In 1844, a small gymnasium was built between Seminary Hall (The Sem) and the Boarding House (Old Metcalf Hall). It is believed to be the first freestanding gymnasium built at any women's college in the United States.
Physical education has been important at Wheaton since its founding. Beginning in 1835, students were required to perform calisthenics, rhythmic movements, breathing and other exercises, and to walk every day in good weather. By the end of the 19th century, gym classes included archery, croquet, tennis, riding, bowling and basketball.
In 1849, a new Seminary Hall (now the north wing of Mary Lyon Hall) was built next to the little gymnasium. Between 1869 and 1878, the building served as the Seminary's Library.
In 1878, Mrs. Wheaton funded additions and renovations to Seminary (Mary Lyon) Hall and to the Trinitarian Congregational Church. She donated the gymnasium to the Church, where it was added to the southeast corner, parallel to the main structure, and served as "a neat and attractive chapel". It was dedicated on 18 September 1878 with a sermon by the Rev. H. K. Craig of Falmouth, a former pastor of the church, and a prayer by the Rev. W.N.T. Dean, the acting minister.
In 1882, Mrs. Wheaton offered to make more substantial changes to the Trinitarian Congregational Church and the attached Chapel. Plans drawn by Worcester, MA, architect Stephen C. Earle called for the small structure to be moved again, this time so that it was at right angles to the main sanctuary, with an additional building connecting the two. The interior walls and ceiling were painted in new (though unnamed) colors. The connecting building contained a "parlor or infant class-room, fifteen by twenty feet, connecting by wide doors with the chapel; a small vestibule on the east connects with the chapel and parlor, and a larger porch or vestibule with a western entrance." The renovated Chapel was reopened and rededicated with a special service on Sunday evening, 26 November 1882, at which the pastor spoke from the text "At the gate which is called Beautiful" (Acts 3:1-10). The main sanctuary was completed in December 1882.
Designed by architect F. Frederick Bruck of Cambridge (an associate of Walter Gropius) at the urging of Professor of Philosophy Holcombe M. Austin, 1963 renovations and additions to the Church included moving this little building for the third time, to its present location facing Pine Street at the southern end of the Sunday School addition; it is now called Sweet Hall. During one of these moves, the cupola was lost, but the barrel ceiling and front pediment, columns and porch survive from the original design.
Sweet Hall was named for the family from which the property was purchased.
[For further information regarding the structure of the Trinitarian Congregational Church and its Chapel, see D. Hamilton Hurd, History of Bristol County, Massachusetts..., Philadelphia: J.W. Lewis & Co., 1883, pp. 614-615]