Poundon used to be a hamlet of Twyford parish, and so does not have its own parish church. Two finds suggest that Poundon dates back to the medieval period, such as the 14th century green-glazed jug found in pieces during the digging of foundations for a garage at 8 Poundon Village. It has since been reconstructed at Buckinghamshire County Museum. A 15th century lead medallion was found on the ground surface 350m northeast of Lower Farm. It had the royal coat-of-arms on it and seems to date to about AD 1405.
Poundon House and garden walls are listed and the formal garden is also a registered garden. The house was built, according to the plaque on the front, in 1908. The gardens include a formal parterre, a walled kitchen garden and an ornamental pond. The house was used by the Special Operations Executive as a communications centre during the Second World War, presumably in partnership with the radio transmitter on Poundon Hill.