iistickboy
commented
3 years ago My NaNoGenMo 2020 project has been completed with an hour to spare!
"Sublime Text" is 50,889-words generated by a combination of GPT-2 and GPT-3.
Link to everything here: https://github.com/iistickboy/NaNoGenMo-2020
My project will pick up where my NaNoGenMo 2019 project left off, utilizing GPT-2 and GPT-3, the language models developed by OpenAI.
Last year, thanks to Max Woolf, I learned how to fine-tune two different GPT-2 language models with Google Colab. I created "Writing Prompt Prompter" trained on thousands of writing prompts posted on /R/WritingPrompts and "Writing Prompt Responder" trained on thousands of responses to writing prompts on /R/WritingPrompts.
After running those language models for weeks straight, I generated a 50,000-word story collection, with a series of stories generated by "Writing Prompt Responder" in response to story prompts generated by "Writing Prompt Prompter." The stories were spooky and lovely.
Late in 2019, Nick Walton created AI Dungeon, a text-based video game that used the GPT-2 language model as its storytelling engine. Walton shared his AI Dungeon code for free through Google Colab and I spent a few amazing weeks running the early version on my laptop. Walton and his team have since developed a standalone AI Dungeon app.
In June, OpenAI announced the creation GPT-3, the third generation of its AI language model. The company said it trained GPT-3 with a massive selection of datasets, including the Common Crawl corpus (around one trillion words gathered through eight years of web searches), two “internet-based books corpora," and the English-language version of Wikipedia.
I can't access GPT-3 on my own, but AI Dungeon now runs on the powerful new language model. I could never code something as elegant as AI Dungeon and I could never replicate the millions of training hours its users have spent on the game.
So for my NaNoGenMo project this year, I will use AI Dungeon as my interface to generate 50,000 words. I will feed AI Dungeon my favorite computer-generated writing prompts and writing prompt responses generated during NaNoGenMo 2019. AI Dungeon will continue where those stories left off last year.
It will be like a game of Exquisite corpse played between GPT-2 and GPT-3.
I will keep an eye on Tra38's project that also uses AI Dungeon.