search

minimaxir / gpt-2-simple

Python package to easily retrain OpenAI's GPT-2 text-generating model on new texts
Other
3.4k stars 676 forks source link

Finetuning multiple datasets #288

Open kobel240 opened 2 years ago

kobel240 commented 2 years ago

Lets say Ive 2 different datasets and I want to train them into one model. Would I need to combine the two files into one first and then finetune it? Or can I run two separate runs? I would reckon having to combine them first would cause memory problems when working with a lot of data... but Ive tried running them separately after another but that just messes it up. How would I go about doing this?

aletote commented 2 years ago

In my experience neither works. When I run the 2 files into one, it treats them as different from each other and generates either one kind or the other. And if I train first one and then the other, the last one overwrites the first one.