Gustavo Petro, a leading leftist, was addressing tens of thousands of supporters in Bogota’s main square.
He was sacked by Colombia’s top prosecutor last month over the alleged mismanagement of the capital’s rubbish collection service.
He has appealed against the decision.
Speaking from a balcony of city hall on Plaza de Bolivar, in central Bogota, Petro told his supporters they now had a “vital responsibility”.
“If we don’t come to these squares, it will be the end of democracy and our chance for peace.
“That is why we need to gather for the largest protest in the history of Bogota – which will be permanent – on the day that the prosecutor ratifies his arbitrary decision,” he said.
A decision is expected as early as early next week, according to the mayor.
The BBC correspondent at Friday’s protest, Arturo Wallace, said there were fewer than the 150,000 people the organisers had expected, but added that their numbers were close to those who gathered last month after the announcement of the mayor’s sacking.
And this, he said, was no small thing in a city where people are barely beginning to return from the Christmas and New Year holidays.
Colombia’s Inspector General, Alejandro Ordonez, removed Mr Petro last month and also banned him from holding public office for 15 years.
Mr Ordonez said the mayor had violated the principles of the free market and endangered people’s health with a series of changes to the rubbish collection system.
In 2012, rubbish was left uncollected in Bogota for several days due to failures in the system.
Mr Petro has said he is a victim of a right-wing coup, and has called his removal unlawful and politically motivated.
The Colombian constitution gives the inspector general’s office the power to oversee the performances of those holding public office.