The Quarantine Station at Point Nepean was built in 1852. When the gold rush was booming and put Victoria on the map, it was considered to be a mixed blessing. It brought in an increasing number of ships with a greater number of passengers which provided more trade in fresh meat and vegetables. The problem was that these ships were often unsanitary and overcrowded. Colonies were also overcrowded and people infected with disease were entering colonies. In 1852, a ship called the Ticonderoga arrived from Liverpool with 795 passengers and 48 crew on board. The passengers were struck down with yellow fever. 100 died while on-board. While they were put in Quarantine upon arrival, there were a further 70 deaths and a mass improvised grave near the foreshore was created. It also lived through the Spanish flu epidemic, smallpox, typhoid, malaria and many more.

In 1952 it was used to house the Army Officer Cadet School. A lot of cadets have stories of their time living at the Quarantine station. There was the story of a ghost called OLD George roaming the camp at night who was a cook on a ship that was quarantined at the station in the 19th century.

In 1999, it was used to house 4000 refugees from Kosovar who were fleeing persecution and needed shelter. The buildings today remain unoccupied with many debates over what they should be used for.