Swearengen opened the Gem Variety Theater on April 7, 1877, at the corners of Wall and Main streets, to entertain the population of the mining camp with "prize fights" (as was customary with Swearengen's previous establishment the Cricket Saloon, no prizes were actually involved), stage acts consisting of comedians, singers and dancers, and, primarily, prostitutes.
Swearengen recruited his prostitutes in the same manner as it is still carried out in many places today; advertising legitimate stage, cleaning, or waitressing jobs in his theater to desperate young women and advancing them the money for their (one way) trip; then, when they arrived, forcing them into what was essentially indentured servitude as prostitutes. Those who balked were first threatened with demands for repayment of the funds advanced to them for the trip; if that failed, they were threatened with beatings, and if that failed they were beaten and physically forced to submit to Swearengen's demands. Many still resisted, but those who escaped this fate could only find themselves in no better situation, as penniless women with no source of income, alone in a rough and rowdy mining camp, and with the constant threat of Swearengen's men hanging over them. Many sickened and died from lack of proper nutrition and shelter, many others committed suicide.
In any event, the Gem prospered, bringing in an average of $5,000 a night, even reaching as high as $10,000. Swearengen forged alliances with many of Deadwood's most prominent citizens, buying himself immunity from legal or other problems. His immunity even extended to the notoriously upright and incorruptible Marshall Seth Bullock, who did not have the political clout to extend his campaign to clean up the town as far as the lower regions of Main Street, which remained Swearengen's territory.
The front of the Gem consisted of a bar, and the "theater" area; in the back were the rooms for the prostitutes. Day to day operations were managed a staff including Dan Doherty and Johnny Burns, both portrayed in Deadwood, although the real-life individuals were reputed to be much more brutal to the prostitutes than seen in the series, as was Swearengen himself. Customers also frequently brutalized the women, even to the point of killing them.
The violence in the Gem was not confined to the prostitutes, with the saloon being a frequent site of gunfights between drunken patrons; and in one memorable instance, the memoirs of John S. McClintock reported a prostitute named "Tricksie" having shot a large hole through the skull of a man who astounded everyone by surviving for another half an hour. (This incident is portrayed in an episode of the series, truth being at least as strange as fiction).
The Gem was damaged by fire in the summer of 1879 and repaired, but then very soon destroyed in a major fire that devastated the town on September 26, 1879. Swearengen built an even larger and more grand establishment, reopening in December of 1879 to adulation as the finest theater ever seen in Deadwood.
In 1899, however, the Gem burned down once again, and a penniless Swearengen declined to rebuild and left for Colorado. Despite the Gem's history as Deadwood's longest-lived entertainment institution, its support by so many of the town power brokers over the years, and the glowing tributes in the press after the rebuilt Gem was unveiled, after its demise it was reviled in the press as an evil institution and a town shame.