A recently surfaced research paper has brought the contentious conversation surrounding artificial intelligence into the realm of actual play. Published in July 2024, a paper by graduate student Pavlos Sakellaridis documents his attempts to create a ChatGPT-powered Dungeon Master based off a dataset using the The Sunless Citadel D&D module and transcripts of Critical Role — provided by fan work, not members of Critical Role itself. The experiment has raised questions about the hazy nature of fan works as they related to consent in training Large Language Models.
As advertisements for LLM-generated adventures have become more prevalent, the research was an attempt to evaluate ChatGPT’s ability to not merely create an adventure but “maintain narrative coherence and foster player involvement,” according to the paper. The academic effort was based on a complex slurry of variously licensed information, with some sourced from private companies, some sourced from a group of performers, and other materials sourced from volunteers.
In the body of the paper, Sakellaridis states that he used The Sunless Citadel as “a reference file for ChatGPT when acting as the DM,” citing proprietary descriptions of characters, locations, and pre-written dialogue from the module during play.
The Critical Role transcripts used come from the Critical Role Dungeons and Dragons Dataset (CRD3), which contains nearly 160 episodes of transcription from the first and second campaigns, along with corresponding summaries collected from the Fandom wiki. The dataset, crafted by Revanth Rameshkumar and Peter Bailey of Microsoft was originally made due to their belief that “better abstractive summarization tools to distill information is essential given the ongoing growth of unscripted, multi-person dialogues in entertainment and business scenarios,” according to their paper on the data set. “We hope that CRD3 will support research and development for such tools.”
Unlike for-profit AI research that is trained on the work of professional artists, Sakellaridis’ research was done as a student project and was trained on the fan-based labor. As noted by actual play historian and Polygon contributor Dr. Emily Friedman on Bluesky, the citations within Sakellaridis’ paper show this data has been used in other LLM-based projects (not affiliated with Sakellaridis or his academic research), while much of the work of transcribing these episodes has been done through volunteer fan labor.
The greatest contributions are from Stuart Langridge through the site Kryogenix, which has searchable transcripts tagged by speaker and time. Protections surrounding the Critical Role episodes places these transcripts in a precarious situation, potentially endangering the largest source of actual play research data available amidst ongoing legal battles over the use of copyrighted material to train AI. At the original time of writing, Kryogenix had been down, but has since gone back online. “[Critical Role] loses out BIG TIME if Kryogenix stays down,” said Friedman in a statement to Polygon, as the work by Langridge enables important research into the still-developing medium with little historical or institutional archives.
The use of generative AI has been a point of repeated contention in the tabletop industry and beyond, with the technology’s critics citing its environmental impact and its foundations on exploitative labor from both workers based in the global south and artists whose work is nonconsensually used to train the tech. Wizards of the Coast, along with other companies, has previously come under fire from its audience for the use (both suspected and verified) of generative AI in their products — resulting in explicit statements from the company that AI will not be used in their art, though last year CEO Chris Cocks told VentureBeat that Hasbro is “doing some stuff around AI that’s really interesting.”
Critical Role, however, is much less on the fence about its relationship with artificial intelligence, going so far as to say “Fuck AI” in a recent episode.
Polygon contacted Critical Role, Hasbro, and Sakellaridis for comment, but did not receive an immediate response.
