Many of these cemeteries are now on privately owned land. I have had wonderful luck with asking those property owners to go view the cemetery and many of the headstones survived albeit rather damaged and deteriorated. I know in the new England area many of these are marked- many but not all. Is there one in particular that you are looking for? You may be able to find the burial location if you have access to the family bible (or copies of pages) or a general idea. If you have not used findagrave.com it is a wonderful resource- though not all cemeteries are listed- especially the older ones.
One of the problems that you may- or have- encountered is any of the records that were around at that time have more than likely been destroyed in one of the wars fought on our homeland. Often the best source to find burial locations is to search church records, though not many of those have survived as churches were burned often in the revolutionary and civil wars.
As if the destruction of the records wasn't bad enough some of the graves may have been moved.
I agree with Kyla Rogers' points. Findagrave has lots of info but sadly some of the graves are located on private property and the grave areas are not protected and the livestock trample the dickens out of the area (don't get me started on the groundhog population -my hubby once found a skull that the groundhog "kicked out" of his burrow!). I found a graveyard of distant relatives and have been slowly cleaning it out. The private graveyards are the hardest to keep up with and Genweb archives sometimes has more info than Findagrave. Check with historical societies and of course - churches! Good luck!