President Sata, who was being treated in the UK, died in London’s King Edward VII hospital on Tuesday night.
Media said that he died after “a sudden onset [of] heightened heart rate”.
It is not immediately clear who will succeed the president. The issue may be decided by the Zambian cabinet which meets on Wednesday morning.
“It is with a heavy heart that I announce the passing on of our beloved president,” cabinet secretary Roland Msiska said on national TV.
He said that Mr Sata’s wife and son were at his bedside.
“I urge all of you to remain calm, united and peaceful during this very difficult period,” Mr Msiska added.
A devout Catholic, Mr Sata had worked as a police officer, railway man and trade unionist during colonial rule. After independence, he also spent time in London, working as a railway porter, and, back in Zambia, with a taxidermist company.
On the fourth attempt, Mr Sata won presidential elections in 2011. At first he looked as if he would keep promises to tackle corruption and create jobs and prosperity. But his term in office was marred by a crackdown on political opposition and a decline in the economy.
‘King Cobra’
The president’s death comes just days after Zambia celebrated the 50th anniversary of independence from the UK.
He is the second Zambian leader to die in office after Levy Mwanawasa in 2008.
Earlier this month reports in Zambia said that President Sata had gone abroad for a medical check-up amid persistent speculation that he was seriously ill.
After he left the country, Defence Minister Edgar Lungu was named as acting president.
Vice-President Guy Scott has regularly stood in for the president at official events. But he is of Scottish descent and his parents were not born in Zambia, so he may fall foul of a constitutional clause on parentage which would nullify his candidacy.
Known as “King Cobra” for his venomous tongue, Mr Sata was elected Zambia’s president in 2011, defeating the then incumbent Rupiah Banda whose party had been in power for 20 years.
He has rarely been seen in public since returning from the UN General Assembly last month, where he failed to make a scheduled speech. (BBC)