Throughout the 15th and 16th centuries, the travertine that covered the Colosseum was removed for use in other constructions, such as the Barberini Palace and the Port of Ripetta. There is a well-known Latin proverb that says "Quod non fecerunt Barbari, fecerunt Barberini" (What the Barbarians did not do, the Barberini did). It was also used for burning in order to obtain lime. This plundering of stones continued until 1749, when Benedict XIV consecrated the monument as a sacred place in memory of the martyrs who were executed there (although it is believed that most of these were martyred in the Circus Maximus).

During the 16th and 17th centuries, ecclesiastical leaders sought a productive use for the dilapidated shell of the majestic Colosseum. Pope Sixtus V (1585-1590) contemplated turning the building into a woolen factory to employ Rome's prostitutes, although this idea did not prosper due to his untimely death. In the 19th century, on the other hand, efforts were undertaken to stabilize numerous ancient monuments. In 1820, several buttresses were completed as part of a restoration ordered by Pius VII, which are still visible today and which, had they not been built, would have prevented the possible collapse of the building. Throughout the century, consolidation and improvement work was carried out, a process that continues to this day.

Adjacent to the Colosseum was the Meta Sudans, a construction erected by Emperor Domitian between the years 89 and 96. This structure was conical in shape and reached a height of 17 meters, being surrounded by water. Its purpose was to mark the point of intersection of the four districts of the city. In 1936, Mussolini ordered its demolition, arguing that it interfered with the creation of the Via dei Fori Imperiali. One of the last affronts suffered by the Colosseum was its use as a symbol of Italy's wartime conflict by military forces; the part of the building missing in the first image was caused by a bomb that fell during World War II.