Vesey House in High Street was under construction in 1635, being built in brick by John Burges, the Rector of Sutton at the time. It was a two-storey house with stone quoins at the corners and seven regularly-spaced stone window frames on the first floor, presenting a flat regular frontage to the street. The house was built round a central courtyard, making it the grandest house on High Street. Burges built it for his wife Lettice - Lettice was the great-granddaughter of Hugh Harman, the brother of John Harman alias Vesey, Bishop of Exeter.
Thomas Willoughby inherited Vesey House, then known as the Brick House. He was a leading Parliamentarian in the Civil War, and an MP in one of Cromwell’s parliaments in 1656. During the Commonwealth period marriages were conducted by Justices of the Peace in private houses, and many Sutton couples were married in the Brick House by Thomas Willoughby in the 1650s.
The 1674 Hearth Tax shows Mr. John Adis as the owner, paying tax on 16 chimneys (New Hall, with 17 hearths, was the only house in Sutton with more). The Addyes family of Moor Hall owned the house in the first half of the eighteenth century, a period when many of the big houses in High Street were having a face-lift to give them a smart Georgian frontage. This made Vesey House look old-fashioned, but when it was modernised a different style was preferred - an extra storey was added as well as three two-storey bays, the whole frontage was rendered and a large stone acorn decorated the apex.
Hacketts succeeded Addyes as owners of Moor Hall and Vesey House - it was known as Acorn House in 1820 when new tenants moved in. The house was divided into two, Dr. Pearson living in one half and Mrs. Bracken in the other - they were brother and sister. Later the three Bracken sisters lived there, and Agnes Bracken the artist and historian purchased the house in 1860 from the Hackett estate. The diarist Sarah Holbeche noted in 1868 that Miss Bracken was upset because the stone acorn at Acorn House had been taken down as unsafe.
Early in the twentieth century the house was renamed Vesey House and given a new front. The bay windows were removed, except for the ones on the first floor, a smooth white rendering was applied with decorative panels of a squirrel and a deer, and there was a new central front door. In the 1970s shops occupied the pavement level, the house itself being occupied as a private residence by Dr. and Mrs. Black. New owners renovated it again in 1990 for use as offices.