Matilda married Richard in Richmond on 23 September 1818. They had 15 children: Samuel, Eliza, William Henry, Sarah Jane, Richard, Alexander Fulton, Anne Eliza, Robert James, the Rev. George Hutcheson, Matilda Frayser, Mary Janet, Isabella Wilson, Douglas Plumer, Ella B., and Emma B.
Their son George Hutcheson Denny's son, also named George Hutcheson (1870-1955), became President of Washington and Lee College (1902-1911) and the University of Alabama (1912-37, 1941-42.) Bryant-Denny Stadium in Tuscaloosa is named after him.