A building has been on the site from the 11th century and it has, for all this time been the home of the Croft family and Croft baronets. The Croft family were closely linked to their neighbours the Mortimers of Wigmore and later Ludlow. The Battle of Mortimer's Cross took place on Croft lands nearby in 1461. The present building originated as a castle in the 14th century and has been much altered since. The Castle was restored after slighting in the Civil War. It now consists of a stone quadrangular manor house with a small castellated round tower at each corner and a small square tower flanking the north side. The castle is under the care of the National Trust and members of the Croft family still live within it.