Julian Castro skyrocketed from a member of the San Antonio City Council, to mayor, then became Secretary of Housing and Urban Development under President Barack Obama.

Now, Castro is back home in San Antonio and questions are already swirling about what he does next.

Castro returned to the Alamo City on January 21. His wife and two children came back to the home they kept while they were in Washington D.C.

"This is the first time, as a family, that we've been separated, so I've been coming back and forth every couple weeks,” Castro said. “It's neat to finally be in one place.”

"We left a family of three and came back a family of four," said Erica, Castro’s wife.

Cristian Julian was born in the nation's capital and just celebrated his second birthday. Big sister Carina is about to celebrate her eighth.

In 2014, Julian Castro resigned as Mayor when President Obama asked him to serve as Secretary of Housing and Urban Development. Now, 2 1/2 years later, Castro returns to a city, he says, has changed.

"In my time at HUD, I had the chance to visit 100 communities in 39 different states. And I'm as proud and impressed by San Antonio now as I was then,” Castro noted. “The city still has to make a lot of improvements. In some ways, we've gone backward on stuff. I think that we've probably lost a little bit of momentum in terms of breaking out of this city and getting the right kind of economic development attention in the nation. We need to step that up.”

However, don't look for him to step into an election anytime soon.

"Right now, I'm going to mostly leave politics alone. I'm not saying that I won't get involved in anything, but I'm definitely not running for anything in 2018, in the next cycle. It's extremely unlikely that I will run for governor or senator," he said.

Instead, Castro says he will concentrate on finishing his memoir, serving on boards, and speaking.

His wife Erica will be the principal bread winner for now. She's the math coach for Harlandale Independent School District.

"I think that was the hardest part of moving to Washington D.C., to put my professional career on hold. But I'm back to work and I'm enjoying it," she said.

In a Facebook post from last week, as Castro was packing up his HUD office, he said that we'll still hear from him.

"I'm going to use my voice because I think over the last several years I've learned a lot about policy. I've raised my profile nationally and I have a voice to use. I'll stay active when issues come up, whether it's education or immigration or healthcare," he said.

Castro and his brother, Congressman Joaquin Castro, boycotted Donald Trump's inauguration, but he hopes that, for the sake of the country, President Trump is successful.

"There were a lot of folks voted for Trump because they wanted to blow up Washington,” Castro said. “We need to keep the spirit of making tremendous change to Washington but do it in a positive way instead of a negative and disrespectful way.”