search

minimaxir / gpt-2-simple

Python package to easily retrain OpenAI's GPT-2 text-generating model on new texts
Other
3.4k stars 676 forks source link

Out Of Memory On Google Colab When Training With Big Dataset #257

Open nikhilanayak opened 3 years ago

nikhilanayak commented 3 years ago

I used gpt-2-keyword-generation to take my dataset and tokenize it. In the end, the file was about 700MB. When I try to train with any model size, the colab notebook runs out of memory. I know my dataset is pretty big, but is there anything I can do to get away with it?

777yeet commented 3 years ago

Not on free tier, as far as I know. They used to give extra memory if your instance crashed but they don't now. You could try and use the TPU-edited version.