Jolidee Matongo had been campaigning with Mr Ramaphosa before the accident.
His vehicle collided with a van while trying to avoid a pedestrian who had run on to the road, local media report.
The pedestrian and the other vehicle's driver also died, while Mr Matongo's two bodyguards are being treated in hospital for their injuries.
Mr Matongo only became mayor on 10 August following the death of his predecessor from Covid complications.
Local media report he was the son of a Zimbabwean migrant, and had been a political activist in South Africa since the age of 13, campaigning against apartheid - a legalised system of racism that was in place at the time.
He was a member of the governing African National Congress (ANC), and was expected to be at the forefront of the party's campaign to win Johannesburg - South Africa's commercial capital - in local government elections due on 1 November.
In his tribute to the mayor, President Ramaphosa said: "It is hard to comprehend this tragedy, given the vitality and passion with which Mayor Matongo interacted with me and residents of Soweto so shortly before his death.
"Nothing could prepare any of us for this sudden loss, which has deprived our nation's economic centre of its second executive mayor in two months."
The premier of South Africa's Gauteng province, David Makhura, said he was shattered by the mayor's death.
"He served as a councillor for many years before being elected as the mayor of Johannesburg," Mr Makhura's office added in a statement.
Johannesburg is the main city in Gauteng, the economic hub of South Africa.
Former Johannesburg mayor and opposition ActionSA leader Herman Mashaba said he and Mr Matongo had had sharp political differences, but he would always "cherish" their "special relationship".