A great fire broke out in the castle in 259 AL, which was a serious blow to House Targaryen. The fire left the castle ruined and resulted in the death of King Aegon V Targaryen, his eldest son and heir Prince Duncan and Lord Commander Ser Duncan the Tall. It is sometimes blamed on one of Targaryen's attempts to bring dragons back by hatching ancient dragon eggs.

In the eyes of Prince Aerys II and his sister-wife Rhaella, the tragedy at Summerhall either heralded or directly contributed to Rhaegar Targaryen's birth. Prince Rhaegar grew up with a complex fascination for Summerhall. Here he was happiest, yet also saddened by the memory of the tragedy. He liked to visit the place himself and sleep in the ruined hall of Summerhall beneath the moon and the stars.