Instructions for fine-tuning pre-trained models

facebookresearch / fairseq-lua

Facebook AI Research Sequence-to-Sequence Toolkit

Other

3.74k stars 616 forks source link

Instructions for fine-tuning pre-trained models #139

Open y3nk0 opened 4 years ago

y3nk0 commented 4 years ago

Is there any chance that we can have a full description on how to fine-tune pre-trained models (for example in machine translation)? I've managed to continue training on a much smaller dataset (by using the pre-trained dictionary) but the results are disappointing. The model gets kind of "broken" (it mostly outputs "unk"). Am I missing any step? Should we update the bpecodes as well? Thank you!

nikitacs16 commented 4 years ago

I'm facing the same issue. @y3nk0, were you able to resolve it? @myleott @michaelauli could you advise regarding the same?