The Pentagon said it was transitioning to the "next phase of the campaign" but did not give details.
Some 2,000 troops have helped rid much of north-eastern Syria of IS, but pockets of fighters remain.
It had been thought defence officials wanted to maintain a US presence to ensure IS did not rebuild.
There are also fears a US withdrawal will cede influence in Syria and the wider region to Russia and Iran.
Trump's Syria troop removal prompts questions
After the caliphate: What next for IS?
Both the Pentagon and the White House statement said the US had started "returning United States troops home as we transition to the next phase of this campaign".
The Pentagon said it would not provide further details of what that next phase is "for force protection and operational security reasons".
The White House said the US and its allies stood "ready to re-engage at all levels to defend American interests whenever necessary, and we will continue to work together to deny radical Islamist terrorists territory, funding, support and any means of infiltrating our borders".
President Trump said it was time to bring troops home after their "historic victories".
What has been the reaction?
Israel said it had been told the US had "other ways to have influence in the area" but would "study the timeline [of the withdrawal], how it will be done and of course the implications for us".
Russia's Foreign Ministry spokeswoman Maria Zakharova said on state-controlled Channel One TV that the US decision could result in "genuine, real prospects for a political settlement" in Syria.
Pulling troops out of Syria had long been promised by President Trump.
But the announcement may have taken some of his own officials by surprise.
Last week Brett McGurk, special presidential envoy for the global coalition to defeat IS, told reporters at the state department: "Nobody is saying that [IS fighters] are going to disappear. Nobody is that naive. So we want to stay on the ground and make sure that stability can be maintained in these areas."
The state department abruptly cancelled Wednesday's daily briefing after the withdrawal was announced.
One of Mr Trump's supporters, Republican Senator Lindsey Graham, who sits on the armed services committee, called it a "huge Obama-like mistake".
In a series of tweets, he said IS was "not defeated", and warned withdrawing US troops puts "our allies, the Kurds, at risk".
Turkey this week said it was preparing to launch an operation against a Kurdish militia in northern Syria, which has been an ally of the US in its fight against IS.
A UK government spokesman said these developments "do not signal the end of the Global Coalition or its campaign" against IS, and the UK will "remain committed" to ensure IS' "enduring defeat".
"Much remains to be done and we must not lose sight of the threat they pose," the UK statement said.
"As the situation on the ground develops, we will continue to discuss how we achieve these aims with our Coalition partners, including the US."
Chaotic US policy?
President Trump's decision reverses the official lines of both the Pentagon and the State Department and it places Washington's Kurdish allies in greater jeopardy.
US ground operations in north-eastern Syria involve some 2,000 troops, maybe more, and a network of bases and air strips has been established. But to what strategic end?
IS is well on the way to being defeated. Syria's President Assad remains in place. If the goal now is to contain Iran or Russia's rising influence in the region then 2,000 troops strung out across a vast swathe of territory may be too small a force to do this.
Their presence though does give the US "skin in the game". And many will see this decision as yet another indication of the chaos and uncertainty surrounding US policy towards this crucial region.
What is the US presence?
US troops have largely been stationed in the Kurdish region in northern Syria.
A partnership with an alliance of Syrian Kurdish and Arab fighters, known as the Syrian Democratic Forces, is credited with playing a major role in the virtual elimination of IS after it overran large swathes of Syria four years ago, imposing brutal rule on almost eight million people in the country and neighbouring Iraq.
Why the battle for northern Syria matters
Who are the Kurds?
However, the militant group has not disappeared entirely. A recent US report said there were still as many as 14,000 IS militants in Syria and even more in neighbouring Iraq - and there is a fear they will shift to guerrilla tactics in an attempt to rebuild their network.
But the partnership between the US and the Kurds has enraged neighbouring Turkey, which views the Kurdish YPG militia - the main fighting force in the SDF - as an extension of a banned Kurdish group fighting for autonomy in Turkey.
On Monday, Turkish President Recep Tayyip Erdogan said his country might soon start a new military operation against the YPG in Syria.
Mr Erdogan added that he had discussed his plan with Mr Trump by telephone and that he had given a "positive response".
The US announced on Tuesday that it had agreed to sell $3.5bn in missiles to Turkey to "increase the defensive capabilities of the Turkey military to guard against hostile aggression and shield Nato allies who might train and operate within Turkey's borders".
Russia said on Wednesday it would continue with its own sale of an advanced missile defence system to Turkey.
In addition to the northern deployment, US forces are also helping fight IS in the last pocket of territory it controls in the south-east of the country.