Mr Mandela, 95, led South Africa’s transition from white-minority rule in the 1990s, after 27 years in prison.
He had been receiving intense home-based medical care for a lung infection after three months in hospital.
In a statement on South African national TV, Mr Zuma said Mr Mandela had “departed” and was at peace.
1918 Born in the Eastern Cape
1943 Joined African National Congress
1956 Charged with high treason, but charges dropped after a four-year trial
1962 Arrested, convicted of incitement and leaving country without a passport, sentenced to five years in prison
1964 Charged with sabotage, sentenced to life
1990 Freed from prison
1993 Wins Nobel Peace Prize
1994 Elected first black president
1999 Steps down as leader
2001 Diagnosed with prostate cancer
2004 Retires from public life
2005 Announces his son has died of an HIV/Aids-related illness
“Our nation has lost its greatest son,” Mr Zuma said.
The Nobel Peace Prize laureate was one of the world’s most revered statesmen after preaching reconciliation despite being imprisoned for 27 years.
He had rarely been seen in public since officially retiring in 2004.
“What made Nelson Mandela great was precisely what made him human. We saw in him what we seek in ourselves,” Mr Zuma said.
“Fellow South Africans, Nelson Mandela brought us together and it is together that we will bid him farewell.”
UK Prime Minister David Cameron paid tribute to Mr Mandela, saying “a great light has gone out in the world”.
Earlier, the BBC’s Mike Wooldridge, outside Mr Mandela’s home in the Johannesburg suburb of Houghton, said there appeared to have been an unusually large family gathering.
Among those attending was family elder Bantu Holomisa,
A number of government vehicles were there during the evening as well, our correspondent says.
Since he was released from hospital, the South African presidency repeatedly described Mr Mandela’s condition as critical but stable.
Born in 1918, Nelson Mandela joined the African National Congress (ANC) in 1943, as a law student.
He and other ANC leaders campaigned against apartheid (white-only rule).
He was sentenced to life imprisonment in 1964, but was released in 1990 as South Africa began to move away from strict racial segregation.
He was awarded the Nobel Peace Prize in 1993 and was elected South Africa’s first black president in 1994. He stepped down after five years in office.
After leaving office, he became South Africa’s highest-profile ambassador, campaigning against HIV/Aids and helping to secure his country’s right to host the 2010 football World Cup.
He was also involved in peace negotiations in the Democratic Republic of Congo, Burundi and other countries in Africa and elsewhere.