Academy Street is named for an early east side institution. The Stoughton Academy and Business Institute was organized in 1888 by its principal, K.A. Kasberg. The school provided a college preparatory program as well as business, music and farming courses. In 1889 a large, three-story brick and stone building was erected at 811 Ridge St., one of the highest points in the city. The school appears to have been affiliated with the Norwegian Lutheran Synod from its earliest years; the synod took over its direction in 1895. In 1900, a fire partially destroyed the building and the school relocated to Albion, Wisconsin. The structure was later rebuilt and used as the Stoughton Surgical Hospital and Sanitarium. The hospital was founded by Michael Iverson, a native of Norway who arrived in Stoughton in 1891. The Iverson family and the hospital staff all resided in the hospital building during its earliest years of operation. The hospital was purchased by the city in 1924 and was renamed the Stoughton Community Hospital. The original Academy building was razed in 1958.

Stoughton Academy, from the school catalogue of 1891 - 92.

Situated on a hillside just outside the district, the two-story, cream brick house at 924 Ridge St. dates from the early 1870s. It was built by Steener Halverson, who burned the brick in a kiln on this property. Halverson was a farmer, and was also a partner in a Stoughton brickyard. Andrew and Clara Asbjornson resided here from the 1930s until the 1980s. The original Halverson farm is now part of the Stoughton Community Hospital and the Skaalen Sunset Home.