As with many early Grand Rapids settlers, Belknap was born in New York in 1846, migrating west with his family to Grand Rapids in 1855. His parents were James A. and Mary A. Belknap. He was the second living son among 7 children listed in the 1860 Census. That year, at 16 he joined the military, rising through the ranks. He was made First Lieutenant by special orders from General P.H. Sheridan due to gallant service in the Battle of Chicamauga in the Civil War, and then Captain in January 1964 due to service at Chattanooga. He was with General Sherman in the "March to the Sea." He was known by the name Captain for the rest of his life and was locally known as an expert on the War.

After leaving the service in 1865, Belknap lived on a farm in Sparta until 1871. On Dec. 26, 1866 he married Chloe M. Caswell; they had four daughters. Chloe died

After 1871 the Belknaps moved to Grand Rapids where he became a blacksmith. As he prospered, he became more involved in public service, first as a fireman, then in several city positions including Mayor in 1884.

Today Belknap is most often remembered as an author of publications which inform the reader of the early history of Western Michigan and the Grand Rapids area.