Joseph Lewis Chaffey was born 25 December 1855 in Grand River Wharf, Prince Edward Island to parents Marie Swallow and George Chaffey. He worked as both a labourer and a fisherman. He died in Little Pond, Prince Edward Island, in 1944 at the age of 88. It should be noted that Grand River is an old name for the Boughton river, and it is not to be confused with present day Grand River, Prince Edward Island.
Joseph had 11 siblings, including Oscar Chaffey, Elizabeth "Lizzie" Mitchell, and 9 others.
He married Flora MacDonald in 1881 at the age of 25. He was initially a Baptist, but became a Presbyterian after marriage. Together they had nine children: Lydia Chaffey, Elizabeth "Lizzie" M Chaffey, Margaret "Maggie" G Chaffey, Gertrude Chaffey, Ethel I Chaffey, Florence Chaffey, Mabel M Chaffey, Elsie Chaffey and Samuel Chaffey.