International Golf Federation president Peter Dawson confirmed Monday that the world No. 3 had withdrawn, citing concerns about the Zika virus.
Last month, Spieth, 22, said he was "uncertain" about taking part. Seen by many as the natural successor to Tiger Woods, his dazzling displays last year announced him as golf's brightest new star and propelled him to the top of the rankings.
Dawson was speaking at a news conference before this week's Open Championship in Scotland, where the game's leading players have assembled for the third major tournament of the golf season.
Spieth's decision means none of the world's top four will be involved when golf returns to the Summer Games for the first time since 1904.
The IGF released a list of 60 eligible men, which is led by world No. 5 Bubba Watson of the U.S. and includes only four of the world's top 10 players.
The women's list of 60, in comparison, features all the top 10. South Africa's 39th-ranked Lee-Anne Pace was the only female professional to say that she would withdraw ahead of Monday's announcement.
While the symptoms of Zika -- which include a rash, headaches and joint pain -- are not severe, the virus has been linked to microcephaly in newborn babies and some cases of the muscle-weakening disease Guillain-Barré syndrome in adults.
Last month, Brazil's new health minister told tourists and athletes that the risk of catching Zika in Rio was "almost zero."
"We are here to put at ease the minds of all residents and tourists coming to the games," Ricardo Barros said at a news conference.
He cited a study by Cambridge University that concluded there was a very low chance that any of the expected 500,000 foreign tourists would get the virus, which was detected in Brazil last year.
However, that has not assuaged the concerns of the world's leading golfers, many of whom have young families or are planning to start them.