The son of Gladys Churchill Robertson and A. Willis Robertson, Pat learned very early in life that he had a responsibility to his family lineage. The Robertson family traces its roots for more than 1,000 years from the Norman conquest of Britain to English kings and founders of the Virginia Colony. Pat is a direct descendant of John Woodson, who joined the Jamestown Colony in 1619.

Three prominent people in Pat’s ancestry are also important in United States history. Two U.S. presidents, William Henry Harrison and Benjamin Harrison, and Benjamin Harrison V, a signer of the Declaration of Independence, are relatives on Pat’s mother's side of the family. Pat is also a distant descendant of John Churchill, the 1st Duke of Marlborough (Winston Churchill is also a relative) on his mother’s side.

The Willis family founded Fredericksburg, Virginia. Pat’s great-grandfather, “King Ab” Willis of Orange County, Virginia, was a key player in that effort.

Pat learned a great deal about life and liberty from his father. He credited an incredible work ethic, frugality, integrity and the understanding of what it means to be a public servant from him.

Absalom Willis Robertson, Pat’s father, was an avid athlete as he played college football, held the Virginia state hammer throw record, was an accomplished tennis player and was passionate about hunting and fishing. His devotion to fish and game earned him the title of “Sportsman of the Year” by Field and Stream Magazine.

A. Willis Robertson entered public life before Pat was born, serving as Commonwealth Attorney for Rockbridge County, Va. He then served as a state senator before eventually serving as Commissioner of Game and Inland Fisheries for the State of Virginia.

When Pat was two years old, his father was elected to the United States House of Representatives where he served for 14 years. In the House, A. Willis Robertson was a member of the powerful Ways and Means Committee, a group commissioned to oversee and guide all bills regarding taxation. While on this committee he served as the principal author of the “pay as you go” taxation plan that still yields a harvest of payroll deductions for the Federal government each month.

In 1946, Pat’s father was elected to fill the U.S. Senate seat of the aging Carter Glass. He served his country in the Senate for 20 years, becoming one of a handful of senior conservative Southern Democrats who controlled much of the business of the U.S. Senate.

Pat said of his father, “He lived humbly and frugally. He never forgot that he was a servant of the people and that his private and public life must be above reproach.”

Both of Pat’s grandfathers were Baptist clergymen. James Robertson, Pat’s first Robertson ancestor, who came to Virginia in 1695, was an Anglican clergyman. Several of his Virginian Churchill and Gordon ancestors were also clergymen.

In 1954, at the end of Pat’s second year of Yale law school, he married a lovely young woman from Columbus, Ohio, named Dede Elmer. Dede was a graduate of Ohio State University and was studying for a master’s degree in nursing at Yale. Dede had been a fashion model and beauty queen before being named runner-up in the Miss Ohio State Contest.

They were married 67 years before Dede predeceased Pat on April 19, 2022, at the age of 94. Their marriage was a true partnership and Dede was a tremendous source of wisdom and encouragement in all of the various ministry activities and decisions that were to come. Together they had four children: Timothy Brian Robertson of Virginia Beach, Virginia; Elizabeth Robertson Robinson of Dallas, Texas; Gordon Perry Robertson of Chesapeake, Virginia; Ann Robertson LeBlanc of Portsmouth, Virginia; plus 14 grandchildren and 24 great-grandchildren.