The traces of Roman building activities have been found in the foundation of the castle. Its oldest parts are the lower section of the tower castle, the so-called runaway corridor dating from the XIV. century, as well as the adjacent cross-vaulted hall. The chapel built by bishop Orbán Dóczy in 1481 - 1486, fitted to the eastern corner of the castle tower is a rare example of the multi-level castle chapels. The castle went into the hands of the commanders of the castle after the Mohács Disaster (1526). Continuous fortifications were carried out on it, because of the threatening Turkish danger. The largest rebuilding took place on the basis of the plans of the Italian Pietro Ferrabosco between 1561 and 1575. Yet, the Turks occupied the castle on 29th September 1594. However it went back to Hungarian hands four years later, on 28th March 1598. The restoration of the building was made by bishop Ferenc Zichy (1743-1783) who also extended the building to its present form. The latest renewal of the castle and the palace was completed at 1984. A permanent exhibition was opened in June 2004 about Bishop Vilmos Apor, who was shot dead by Russian soldiers in the cellar under the castle in 1945. The exhibition keeps the memory of Vilmos Apor's martyrdom alive, presenting his life and works, his sacrifice and his martyr's death. Bishop Apor's devoted life and martyrdom is presented not only by excerpts from his diary but also with the aid of different relics, documents, music, light effects and various audio-visual means.