Sally Jessy Raphael’s Mansion, Yours for $6.5 Million


Transformed in the early 20th century from a modest country home to a grand Tudor Revival pile, this Dutchess County estate has been home to former talk show host Sally Jessy Raphael since the 1990s. Packed with intricate woodwork and an abundance of rooms, the house is just one of the architecturally intriguing structures on the 25-acre property known as Elmwood Farm.

The estate on the market at 618 Quaker Hill Road in Pawling is, as the road name implies, in the historic settlement of Quaker Hill. The first Quaker meeting house was constructed on the ridge of land around 1764 and still stands. A prosperous community developed, including the Akin family, whose descendants lived on this property until 1997.

John Akin built a home on this land around 1796, and the property descended to Frances Akin Gould and her husband David R. Gould. Unfortunately, a fire in February of 1872 destroyed the house. Local papers reported a complete loss of the furnishings as well, with one estimating the value at $8,000 with “insurance not ascertained.”

a horse and buggy in front of a wood frame house

An undated photo, published in 1913, shows the Akin-Gould house. Photo via Ancient Homes and Early Days of Quaker Hill via Library of Congress

David R. Gould died just a year later, in March of 1873, at the age of 52. Frances Gould carried on the rebuilding of her family home, with some later sources crediting her as rebuilding on the same foundations of the burned dwelling. The new house, architect unknown, was completed by the mid 1880s when local papers reported her spending the summers on Quaker Hill in the company of her daughter, Beulah Gould Branch, and son-in-law.

Beulah had married southerner John Kerr Branch in 1886, and the couple transformed the family summer home into the Tudor fantasy that exists today. Rather than start from scratch, the house built after the 1872 fire was subsumed into a massive new home designed by architect Edward Lee Young. Work began in 1900 with the Pawling Weekly Recorder reporting in June of that year that Mr. Branch was checking in on the progress of the renovation of the residence. Work included “the placing of steel lathing on the outside of the dwelling, which when covered with a stone cement gives the appearance of brown stone.”

An architect based in Manhattan at the time of the project, Edward Lee Young had previous experience as the chief designer for an architectural woodwork and decorating company and contributed articles on renovating and decorating to trade publications in the 1890s. In an 1895 article he recommended that the rooms of a house should be designed to be harmonious and “in scale with the personality of its owner.”

pawling - exterior with massive chimney

The scale at the Gould and Branch summer home was certainly grand, inside and out. The complex exterior includes a dramatic brick chimney, half timbering, oriel and bay windows, ornate vergeboards gracing the gables, finials, and carved figures under some of the windows. A square tower dominates the main facade while a smaller octagonal tower anchors a smaller wing. A porte-cochère marks the main entrance.

Among the wealth of details on the interior is stained glass, leaded glass, stone and wood mantels, wainscoting, and an elaborate stair ornamented with carved animals, just to name a few of the eye-popping features. Additions to the interior seem to have continued for several years. In 1912 a Columbia County newspaper reported that Hudson firm of William H. Traver & Son had recently shipped wainscoting and a door for installation in the Branch’s Quaker Hill home. Hand carved in “the English gothic style” the door was described as having “cathedral glass.” The couple filled with house with art and furnishings from their travels.

Despite the grandeur, it was just the summer home for the couple. Winters were spent in Richmond, Virginia where they would also live on a grand scale. Their Monument Avenue mansion there was designed by John Russell Pope and completed in 1919. Still standing, it opened as The Branch Museum of Architecture and Design in 2005.

carvings of figures and lions

The carvings include the initials “LB,” perhaps a reference to Louisa Branch who was born in 1901

While architect Edward Lee Young was not involved in that house, the couple must have been happy with his work, as in 1906 he presented plans for the design of the interiors of the Akin Free Library located nearby. John Kerr Branch was a board member and on the selection committee for the design, according to the National Register nomination for the library.

In addition to their winter and summer homes, the Branch family would also add an Italian villa to their property holdings, rotating amongst them. While most years of the census found the family recorded in their Richmond home, the 1910 census shows the family ensconced at Quaker Hill. In addition to their three children, John, Zayde and Louisa, and the still living Frances Akin Gould, the household included Charles Washington, a butler, and Lucy Harris, a servant. Both are identified as Black in the census, with 40-year-old Charles born in Maryland and 30-year-old Lucy born in Virginia. Neither appears as living in the family’s Richmond home in the 1900 or 1920 census.

Known as Elmwood since at least 1910, the estate included enough land for a bit of cattle. The cows, primarily Holsteins according to reports of the time, had a rather impressive dwelling themselves. Still on the property is the large barn complex with a quirky Gothic-inspired entrance.

the gothic inspired barn

The property stayed a family summer home through the 20th century. In 1944, Beulah Branch Gould was photographed by Life Magazine, posing in front of her home as part of an article on the Pawling community that found itself thrust into the spotlight as the home of Presidential nominee Thomas E. Dewey.

When in 1997 the house passed into the hands of Sally Jessy Raphael, known for her signature red spectacles and her talk show, it became home to a collector, as seen in the listing photos. The mansion is packed with furnishings and artwork. In addition to adding her own touches, like a hand-painted mural depicting circus life, she restored the original details of the house, including the intricate woodwork of the main stair.

With 15,000 square feet and 43 rooms, the residence certainly isn’t lacking in space. There is an abundance of entertaining capacity on the main level along with 17 bedrooms and nine full and four half baths.

dining room with wood ornamented ceiling, wainscoting and a mantel with inset artwork

large sunroom with wood floor and ceiling

In the dining room the woodwork extends from floor to ceiling, with paneled wainscoting, a built-in window seat, and a columned mantel. The later surrounds one of 10 fireplaces in the house. If the dining room still isn’t large enough, a massive glass enclosed porch has room for multiple dining tables and access to the garden.

Not surprisingly, the bedrooms are spacious with room for sitting areas. One has double doors opening into an adjoining room set up as a chapel, perhaps inspired by the large stained glass window.

aerial showing multiple buildings on property

Guests can also be hosted further afield in two caretaker dwellings on the acreage. Additional outbuildings include a carriage house, a yoga studio, greenhouse, and the barn.

Listed with Harriet Norris of Douglas Elliman, the property is asking $6.5 million.

[Photos via Douglass Elliman unless noted otherwise]

stair with intricate carving

parlor with beamed ceiling and wood paneling

music room with tiled floor and white mantel

a billiard room with beamed ceiling and a fireplace

den with vaulted ceiling

doors to sunroom with stained glass

windowed porch nook with loveseat

sunroom wood floor

a large bedroom with a windowed niche

room with stained glass windows and wood floor

a bedroom with paneling and a stone fireplace

a bedroom with a seating area in a windowed niche

a bedroom with floral wallpaper

hand painted circus themed illustrations along several walls

bathroom with white console sinks and delft accent tiles

aerial view showing gardens at rear of the house

exterior showing arched windows of enclosed porch

exterior with complex roofline, and carved figures

pawling - house exterior with massive brick chimney, half timbering and complex roofline

aerial showing the house and barn complex

exterior with timber framing

a barn with a cupola

horses in the paddock

horse in the barn

aerial view of the barn complex

Related Stories

Email with further comments, questions or tips. Follow Brownstoner on Twitter and Instagram, and like us on Facebook.


Leave a Reply

Your email address will not be published. Required fields are marked *