Hardscrabble is at the mouth of Bryson Hollow on the upper reaches of Sanders' Fork. It is the site of the Sanders' Fork Baptist Church and about a mile south of the Shiloh Baptist Church. When a post office was established there in the 1880's, it took the name of Brysonville.