In 1723, when Sherman was 2 years of age, his family relocated from his Newton, MA, birthplace to Dorchester (present Stoughton). As a boy, he was spurred by a desire to learn and read widely in his spare time to supplement his minimal education at a common school. But he spent most of his waking hours helping his father with farming chores and learning the cobbler's trade from him. In 1743, 2 years after his father's death, Sherman joined an elder brother who had settled in New Milford, CT.