Swayne Hall was built in 1857 as a Baptist men's college. Following the American Civil War, it became a part of Talladega College, Alabama's oldest private, historically black, liberal arts college.