Baylor and Frances were 1st cousins as their mothers were sisters.

The house Baylor built burned within several years of his death, most likely before 1775. With the uncertainties of the pre-American Revolution times, Frances rebuilt the house quickly using materials from some of the earlier buildings still standing up on the fort hill and 90,000 bricks made using the kiln on the property. Her family may have assisted, as her father Humphrey Hill may have been living. This house was located approximately 10-20 feet further away from the riverbank than the first, but close to the original foundation. This was the basic house that stands today. The house was one room deep and when one entered the house from the riverside, the stairs were on the left of the hall and the door to the twenty-foot square parlor was in the middle wall on the right. Immediately left of the entrance, a door led into the chamber (parents bedroom), which measured twenty by sixteen. The hall was fifteen feet wide and had a fireplace that made it usable as a sitting room in cold weather. There was a flight of steep steps called the “traveler’s stair,” on the far left of the residence which led to a half story second floor bedroom from an outside door on the riverside. There was also an entry from the chamber to this stair, which made it easy for parents to check on the travelers and the children, and for children to have an easy access to their rooms without bringing mud and assorted cluttering objects into the front hall. The landside had a modest entrance to the hall. Upstairs were three bedrooms and a hall. There was a basement that held a dining room with a fireplace and a room to prepare food brought in from the separate kitchen dependency. There was a bedroom with a fireplace in the attic. The woodwork installed by the builder was plain, probably because of the haste to move from temporary quarters. An alley of locust trees framed the entrance. There were dependencies such as the separate kitchen building and the smokehouse.