For centuries Kadyny has been associated with rich magnates and even with Kaiser Wilhelm II, grandson of Queen Victoria of England, and cousin of Tsar Nicholas II of Russia. In 1415 Kadyny was ruled by the Bażyński family and their legacy has remained in the form of the Bażyński oak tree, the largest and possibly oldest oak tree in Poland.

In 1720 the first building works were undertaken on the Baroque palace by Theodor Schlieben and it was the next proprietor Jan Ignacy Działyński who accomplished the palace’s construction. By the end of the 18th century the coat of arms of the Von Schwerin family had been places on the tympanium of the palace to be joined a century later when Kaiser Wilhelm’s coat of arms was added when he was gifted the palace and estates in 1896. The palace was also home to his grandson Prince Louis Ferdinand and the Russian Princess Kira.