James was born in Massillon, Ohio, south of Cleveland, Ohio where his father worked for the railroad before being transferred to Cleveland. He attended college at Kent State University, served in the US Army where he saw active duty as a medic in Korea, then earned his law degree at Case Western University in Cleveland, Ohio. He married Constance Jo Ann Kozelka on October 25, 1958 and has three children: Charmaine Mary, Michael James, and Brian Keith.