Mayor Michael Signer said he was disappointed the white nationalists had descended on his town and said responsibility lay with Mr Trump for inflaming racial prejudices during his presidential campaign last year.
He said: “I’m not going to make any bones about it. I place the blame for a lot of what you’re seeing in America today right at the doorstep of the White House and the people around the president.”
The clashes broke out at a protest over the decision by the city’s government to remove a statue of a confederate general.
Mr Trump has now issued his first tweet condemning the violence, several hours after the violence began.
Some on Twitter accused the President of not being specific enough in his tweet as he did not specify that he was referring to white supremacists. Several members of the Klu Klux Klan joined in with the protests, with punches being thrown and pepper spray released by those in the crowd.
Mr Trump has repeatedly been accused of having stirred up racial tensions in the country through his rhetoric on immigrants and his vow to build a wall between the US and Mexico.
Officials in Charlottesville declared a state of emergency shortly after 11am local time as the violence worsened.
The clashes are the latest since the city, situated around 100 miles from Washington DC, voted to remove the statue of General Lee from a park.