Sutton Coldfield Manor House was pulled down in the 1520s. It had been more like a castle, with a curtain wall and a gatehouse, and the stone buildings included the Chapel of St. Blaise. In its hey-day in the twelfth century it would have been a grand building. The Lord of the Manor of Sutton did not live there, having castles elsewhere in the country, but Sutton was the centre of good hunting country, and as an occasional hunting lodge as well as a symbol of power and wealth, the Manor House was a stately home.

For sixty years after the Norman Conquest Sutton Coldfield was a Royal manor, and it was during this period that a timber-framed manor house fortified with stone fit for a King was built. Before the Conquest the Manor of Sutton belonged to the Earls of Mercia, and it was on the site of their splendid wooden hunting lodge on Manor Hill that the Norman house was built. In 1126 the lordship of the manor of Sutton was transferred to the Earl of Warwick, who also needed to display his high status through the splendour of his manor house.

Fashions change, and by 1200 the manor house was out of date - time to build a new one with the latest accessory - a broad moat. New Hall was built, and the old manor leased out to tenants. The old manor house fell into disrepair, and was untenanted for most of the fifteenth century. It was probably a ruin when the Marquis of Dorset took most of the stone away for his new mansion at Bradgate Park, near Leicester, and the rest of the stones were put to use by Bishop Vesey in local buildings.

The site of the Manor House and grounds was at the disposal of Bishop Vesey, and he gave the land as a dowry to his niece when she married Thomas Keene. Keene’s granddaughter Ann Gardiner brought the property with her as her dowry when she married Simon Parrott of Maney Hall - he got into debt, but not before he had secured her dowry on his wife, so that it came to her second husband Marmaduke Dawney. The Dawneys occupied it for most of the seventeenth century, then it became part of the Holt Estate of Aston Hall. The farmhouse built on the site c.1700, now a private residence known as The Manor, has been much improved over the years.