John Jay McGilvra was born in Livingston County, New York, in 1827 and moved with his family to Illinois in 1844. After teaching school for several years, he began reading law, was admitted to the Illinois bar in 1853, and entered private practice in Chicago. In 1861, McGilvra was appointed U.S. attorney for Washington Territory, a post he held until 1865, when he returned to private pratice and politics. McGilvra was elected to the territorial legislature in 1866 as a Republican. The most notable accomplishment of his single term was the passage of enabling legislation for a wagon road over Snoqualmie Pass, the first trans-Cascade connection between eastern and western Washington. Following the Northern Pacific Railway's announcement in 1873 that Tacoma would be its West Coast terminus, he joined with other prominent Seattleites in organizing the Seattle & Walla Walla Railroad Company. Even after the failure of his railroad venture, McGilvra lobbied and litigated against the Northern Pacific and remained one of its most vocal critics.

McGilvra, one of the first attorneys in Seattle, spent most of his career in private practice but served a short time as city attorney. In addition to practicing law (he would eventually earn the title of "Judge" as a tribute to his longevity), McGilvra speculated in real estate. He and his wife, Elizabeth, purchased a large tract of land bordering Lake Washington, becoming the first white settlers in what is now the Madison Park neighborhood in Seattle. He built what became Madison Street to link his property with downtown Seattle and organized the Madison Street Cable Railway Company. McGilvra was also an advocate of municipal improvement and civic reform in Seattle, remaining active in civic affairs even after his retirement in 1893. He was a primary backer of the Lake Washington Ship Canal and promoted the concept of bringing Cedar River water to Seattle. He and other community leaders established a Municipal League in 1894, although it was soon disbanded.

McGilvra's son Oliver was a successful Seattle lawyer. Daughter Caroline McGilvra, a philanthropist and avid collector of Native American artifacts, was married to Seattle pioneer Judge Thomas Burke, for whom the Burke Museum of Natural History and Culture is named. John J. McGilvra died in 1903.