Pirating ships, cargoes and their sailors for ransom was a lucrative source of income for the Town and a legitimate one in time of war, when Ryers were licensed by the Crown as Privateers.

Smuggling began when Edward I imposed customs duties on wool to boost the royal revenue. Despite the penalty being death, almost everyone in the area was involved in "owling'- as smuggling was called here, due to the secret owl calls between the men. A specialized lantern was used for secret communication is kept in the Museum. Wool was taken out in return for luxury goods, including spirits, tobacco and tea. There were few convictions as the juries were local and many buildings in Rye were modified with secret cupboards, panels and 'hidey-holes' for the contraband, and secret passages and ways through attics for the smugglers to escape capture.