The church was raised between 1820 and 1825 following the cathedral design of Piotr Aigner, and according to the plans of moving the bishop's headquarters to Suwałki.
The main altar of a classicistical style (mid 19th century) situated on a high plinth in the church is the main place in the temple. In the middle of it there is the painting of Immaculate Mother Maria (by Anna Czartoryska, the 19th century). The second painting - of St. Alexander (a pope and a martyr) is situated behind the plinth hanging on the lines in the altar. It was painted in a romantic style in the first half of the 19th century (the author unknown) and it presents St. Alexander on the cloud with his hands being risen. In his right hand he is holding a martyrdom palm. At his feet there are a tiara and a pope crosier. The bottom of the painting presents a martyr scene.
On both sides of the altar there are the columns with the Corinthian capitals and stripes of Acanthus branches on them, as well as the pilasters. Next to them there are the statues of St. Peter (on the left) and St. Paul (on the right). On the frieze you can see the pope insignia: a double cross, a pastoral and a tiara. Above - there is a frontage with "a window of divine providence" put in the tympanum topped with a cross. On the both sides of the tympanum you can see the statues of St. Maria and St. John. Under the arc of the vault of the church there is a stained-glass window showing the emblem of Suwałki.