In the highest part of Chinchon stands the Castle of the Counts. The Catholic Monarchs granted Cabreja the extensive manor in the south of the province of Madrid, centred on the town of Chinchon. It is likely that, following the custom, the new lords built a castle as a residence and to flaunt their power. At the top of the entrance door you can see the crest of the Counts of Chinchon. The castle was besieged and subject to other military occupations that left it in deplorable conditions and although it was repaired and even rebuilt, the Counts of Chinchon stopped living there in the 18th century.