The Castle is caput of the Barony of Cessford, and the principal stronghold of the Kers/Kerrs, notorious Border Reivers, many of whom served as Wardens of the Middle March.With up to six storeys, two of which were barrel vaulted, and with walls up to 13 feet (4.0 m) thick, it was a formidable place of defence.The angle of the building was enclosed by a single-storey defensive gatehouse, and the whole was surrounded by a barmekin and defensive earthworks, a fact that is corroborated by the record of English troops having to use an escalade to gain access to the castle courtyard during the siege of 1523.After Berwick upon Tweed was captured by Richard, Duke of Gloucester in July 1482, Henry Percy, Earl of Northumberland burnt a number of places in the area.At the end of his campaign, on 22 August 1482 he knighted twenty of his soldiers at the "mains of Sessford.