Magdalena Andersson, was announced as leader on Wednesday but resigned after her coalition partner quit the government and her budget failed to pass.
Instead, parliament voted for a budget drawn-up by the opposition which includes the anti-immigrant far right.
“I have told the speaker that I wish to resign,” Ms Andersson told reporters.
Her coalition partner, the Greens Party said it could not accept a budget “drafted for the first time with the far-right”.
Ms Andersson said that she hoped to try to become prime minister again as a single party government leader.
The speaker of parliament said he would contact party leaders on the next move.
Ms Andersson was elected as prime minister earlier on Wednesday because under Swedish law, she only needed a majority of MPs not to vote against her.
Her election at the head of a minority government followed an 11th-hour deal with the opposition Left party, in exchange for higher pensions for many Swedes. She also secured the support of coalition partner the Greens.
Of the 349 members of the Riksdag, 174 voted against her. But on top of the 117 MPs who backed Ms Andersson, a further 57 abstained, giving her victory by a single vote.
Before MPs backed Magdalena Andersson, Sweden was the only Nordic state never to have a woman as PM.
Becoming the first woman prime minister in Swedish history should have been cause for a night of celebration for Magdalena Andersson, yet the sun had barely set when she handed in her notice.
The complexities of Swedish politics mean we can’t assume we’ve seen the last of her, though. If there’s another prime ministerial vote, Ms Andersson will probably get voted in again. This is because the Green party has promised to support her, despite quitting as a formal coalition partner. But she’d end up in a vulnerable position at the helm of a fragile minority government, and would still have to stick to the right-wing budget already voted on by parliament.
What all this political chaos has underlined is just how divided Swedish politics is right now. We’ll have to wait and see whether voters break the deadlock with a significant shift to the right or the left at next year’s elections.