On the remains of the 17th-century earthen rampart that surrounded the fortress of Penza, a memorial complex was built in 1980, a historical composition titled "The Defensive Rampart of Penza in the 17th Century" (the author of the composition is artist-architect Azariy Mamatkadze). The composition included a fragment of the fortress wall with a watchtower, near which a cast-iron mortar cannon was installed, and opposite the rampart stood a monument to the First Settler. During the 2020 reconstruction, the fragment of the fortress wall was demolished — in its place appeared a walking staircase leading from Kirov Street to the Spassky Cathedral.

The earthen defensive rampart, on top of which ran a wall made of sharply pointed logs, with watch and lookout towers standing at the corners, served to protect the fortress until the first third of the 18th century. In front of the rampart was a moat filled with water, studded with sharp stakes, and scattered along its bottom were "chesnochek" — iron caltrops that wounded the legs of enemy horses.

The city-fortress was raided from the Wild Field — from the southeast — by Crimean Tatars, Kuban Tatars, and Nogai Tatars up until the end of Peter the Great’s reign. In 1717, one of the most terrible raids occurred, which entered history as the Kuban Massacre. All fortresses on Penza land were wiped off the face of the earth, and the defenders were slaughtered to the last man. Only the fortress of Penza stood firm thanks to its location, the courage of the garrison, and its armament of cannons and muskets.