Gus Cannon was born on a plantation at Red Banks. He taught himself to play using a banjo made from a frying pan and raccoon skin. He ran away from home at the age of fifteen and began his career entertaining at sawmills, levee and railroad camps in the Mississippi Delta around the turn of the Century.
While in Clarksdale, Cannon was influenced by local musicians Jim Turner and Alex Lee. Turner's fiddle playing in W. C. Handy’s band so impressed Cannon that he decided to learn the fiddle himself. Lee, a guitarist, taught Cannon his first folk blues, "Po' Boy, Long Ways from Home", and showed him how to use a knife blade as a slide, a technique that Cannon adapted to his banjo playing.
Cannon left Clarksdale around 1907. He soon settled near Memphis and played in a jug band led by Jim Guffin. He began playing in Memphis with Jim Jackson. He met harmonica player Noah Lewis, who introduced him to a young guitar player named Ashley Thompson. Both Lewis and Thompson would eventually become members of Cannon's Jug Stompers. The three of them formed a band to play parties and dances.
In 1914 Cannon began touring in medicine shows. He supported his family through a variety of jobs, including sharecropping, ditch digging, and yard work, but supplemented his income with music. Cannon began recording, as "Banjo Joe", for Paramount Records in 1927. At that session he was backed up by Blind Blake. After the success of the Memphis Jug Band's first records, he quickly assembled a jug band featuring Noah Lewis and Ashley Thompson (later replaced by Elijah Avery). Cannon's Jug Stompers first recorded at the Memphis Auditorium for the Victor label in January 1928. Hosea Woods joined the Jug Stompers in the late 1920s, playing guitar, banjo and kazoo, and also providing some vocals.
Although their last recordings were made in 1930, Cannon's Jug Stompers were one of Beale Street's most popular jug bands through the 1930s. He returned in 1956 to make a few recordings for Folkways Records. In the "blues revival" of the 1960s, he made some college and coffee house appearances with Furry Lewis and Bukka White. He also recorded an album for Stax Records in 1963, following the chart success of "Walk Right In", with his fellow Memphis musician, Will Shade, the former leader of the Memphis Jug Band.