Oliver was the youngest of ten children of John Waymouth (Accountant) and Ann (nee Blackman).   He was educated at the Auckland College and Grammar School.   After two years with an insurance company and a period on a farm, he joined his father as a junior partner in the public accounting firm of Waymouth and Son and carried on the business after the death of his father in 1892.   He was one of the first members of the NZ Institute of Accountants.   After a period as secretary, he became a Director of the Northern Union Steamship Co.   Oliver was a keen yachtsman.   His yachts won several regattas in Auckland and Wellington.   He was a judge at the Auckland regattas and was also well known in cricket circles as a member of the West End Cricket Club.
In 1886, Oliver married Cecilia (Cissy) Nelson, daughter of Charles and Ihapera Nelson.   They had five children.
Cissy's parents managed the Geyser Hotel at Rotorua.