Open racheledelstein opened 1 year ago
That sounds good
Another potentially useful thread here
And here which they linked in the DialoGPT repo
Oh cool yeah I've seen that. Though from what I've been reading now fine-tuning actually doesn't solve the problem since the "fine-tuning process robs [GPT2] of the ability to produce diverse responses, eventually reducing them to dull conversation partners.
Could potentially fine-tune on daily dialogues data or any data I guess. I did find this recent paper on fine-tuning DialoGPT with promotes which is supposed to generate "more novel and diverse responses with a higher BLEU score while training only a small portion of the DialoGPT/GPT-2 parameters". So this could save on training time. I'd be keen to work on this together