Cook's Castle was built around 1174 by Sir Richard Worlsey who was also responsible for the lovely Appuldurcombe House. The castle is nothing more than a modem turreted tower, said to have been built by Sir Richard in order to enrich the view from the windows at Appuldurcombe House.
It was deeply embowered in trees and was a favourite place of resort for picnic parties and for those who climbed it a enchanting views of Appuldurcombe House as well as the Worsley Monument, the Free Mantle Gate, to the East, Shanklin and Sandown and in the distance the white cliffs of Culver.
A quotation from Jenkinson's practical guide to the Isle of Wight (1876)
"Those who continue on the path all the way from Shanklin will reach Cook's Castle by bending to the right soon after the Worsley obelisk comes in sight, and then entering a wood, by a stile at the south-east corner."
During the 1840s George Brannon very fortunately decided to engrave the then crumbling folly as one of his Island points of interest. It shows as can be seen above a very fine looking folly that would have been very enticing a landmark to visit.
Over the next hundred years it can only be assumed that the state of building's crumbling walls became a useful pile of stone ruble for recycling into other estate projects and its final demise was probably sealed by a dramatic act during World War II. On the 16th August a Spitfire Mk 1 - X4016 piloted by Flying Officer H.P.Connor from 234 Squadron after tangling with a Messerschmitt Bf109 had to bail out leaving the Spitfire to spiral down and crash next to the ruins, causing a large explosion and substantial damage over the area of the folly.
Today on St. Martin's Down the position is approximatly marked as no physical remains of the folly excist, by a stone cairn engraved with the words "Site of Cooks Castle".