Give Up the Ghost was formed when Tim Cossar and Wesley Eisold (who was then a roadie for Ten Yard Fight) met up with Azy Relph and Jesse Van Diest in 1998. They recorded a demo tape in 1999, followed by their debut release, a self-titled 7", on Bridge 9 Records under their original name American Nightmare in 2000. After extensive touring and line-up changes, the band recorded their second release 7", The Sun Isn't Getting Any Brighter. This was later combined with their self-titled effort to form Year One, which was released posthumously.
Their debut full-length, Background Music, was recorded for Equal Vision Records in 2001 and was released to much acclaim. In 2003, the band faced a legal battle with a similarly named band from Philadelphia. The band then changed their name briefly to the initials "A.N." then to American Nothing for a brief period. After the other American Nightmare threatened to press charges again, they changed their name to Give Up the Ghost (which was intended to be the name of their second album). Their second full length, We're Down Til We're Underground released on Equal Vision Records, displayed the band experimenting with their sound, with longer songs that didn't follow the typical hardcore songwriting formula they had adhered to in their previous releases.
Some fans of the band have been known to refer to them by their original name.