Federal Census 1860-1870

I can't find where my gg grandfather's family is in those two censuses.I know they didnt move away.I will keep looking today.Also I have been sorting out the Grooms' that I am related to in Chesterfield County.