Downing Street says a return to frontline politics by David Cameron would be a matter for him.
The comments came after reports of Mr Cameron wanting to make a political comeback drew mockery from Labour MPs.
The 52-year-old ex-prime minister has told friends he wanted to return to the front benches two years after leaving Downing Street in the wake of the Brexit vote, according to The Sun.
Mr Cameron would not be against the idea of Cabinet recall in the government of whoever succeeds Theresa May as Prime Minister, the newspaper said.
A spokeswoman for Mrs May said: "Whether he chooses to seek to return to frontline politics or not is entirely a matter for him."
The spokeswoman added the PM and Mr Cameron "often speak".
Shadow education minister Angela Rayner labelled the rumoured return of Mr Cameron as "bizarre".
Ms Rayner, MP for Ashton-under-Lyne, was among a series of Labour MPs who reacted to reports of the former Prime Minister fancying the job of Foreign Secretary.
Ms Rayner tweeted: "Just when you thought politics couldn't get anymore bizarre.
"No David please stay in retirement you caused enough damage last time."
Shadow communities and local government secretary Andrew Gwynne wrote: "God. No. Didn't he do enough damage first time round?? Please spare us all."
Emily Thornberry posted a facepalm emoji in response - an image used to display frustration or embarrassment at a certain situation.
Redcar MP Anna Turley said: "The sense of entitlement is unreal. Please go away and think about what you've done"
Yvette Cooper tweeted: "What, because it worked out so well last time? Man, you ripped up our closest international partnership. By accident. That makes you even worse than Boris Johnson."