In 1724, Peter I gifted his associate Yusupov a plot of land between Bolshaya Proseka (Sadovaya Street), the Nameless Yerik (Fontanka River), Tsarskaya Prospekt (Moskovsky Prospekt), and Proseka (Boytsova Lane).

In the 1730s, there stood a one-story wooden house "with nine windows on a high stone basement," built according to a model project by architect Domenico Trezzini. In the 1740s, a stone palace in the Baroque style was constructed. Behind it was a formal garden with ornamental ponds and canals.

The Yusupov estate on the Fontanka River in the mid-18th century was one of the richest in the city. On the plot stood a luxurious Baroque palace, which in plan resembled a widely spread letter H. The building on high cellars combined a central two-story volume with one-story transverse wings, connected by one-story sections. The estate was separated from the river by a trellis fence. On either side of the central driveway alley leading from the palace to Sadovaya Street were large bosquets, the centers of which were marked by ponds of ornamental shapes connected by a canal. A bridge was thrown across the canal along the central axis of the complex. From the ponds to the edges and corners of the bosquets led additional paths bordered by trimmed greenery. The park did not quite reach Sadovaya Street, and the driveway alley passed between small estate buildings facing the street, belonging to other owners. In front of the L-shaped orangery on the Fontanka, there was a small parterre garden with gazebos. The only known image of the palace from that period survives on the axonometric plan by de Saint-Illier.

In 1790, at the request of the owner Yusupov, the palace was rebuilt by architect Quarenghi. Due to the construction of a through embankment and the completion of the Fontanka riverbank improvements, the palace was reconstructed in the 1790s for Prince Nikolai Borisovich Yusupov by architect Giacomo Quarenghi. The architect preserved the palace walls but removed the Baroque decoration and gave the building a classical appearance. He retained the main volumetric divisions of the building (a raised center and lower wings) but completely changed the decorative design of the facades and interiors in the spirit of the time, resulting in a Neoclassical style. On the river side, the architect added service buildings and organized a new enclosed ceremonial courtyard with an entrance from the embankment. Instead of two ornamental wings, he built an open vestibule in the center of the facade with a ramp leading to it. The corner risalits were connected to the raised central loggia, decorated with Tuscan columns. The opposite garden facade was adorned with a six-column portico on a terrace at the first-floor level, accessed by a wide single-flight staircase. The park was redesigned in the landscape style, changing the shapes of the pond shorelines to more natural forms and creating an artificial island. On the site of the former orangery, Quarenghi planned to build a small pavilion with an open semicircular niche facing the Fontanka, a small study, and a hall on the garden side, separated from the rest of the estate by a fence.

In 1810, the Yusupovs sold the palace to the treasury. It housed the Institute of the Corps of Transport Engineers. Between 1836 and 1840, the fence that encloses the building and the adjacent garden on the Sadovaya Street side was made; it was created by engineers Kolman and Trofimovich.

Sources:

https://www.citywalls.ru/house10422.html

https://ru.wikipedia.org/wiki/Дворец_Юсуповых_на_Садовой_улице