Closed mailong25 closed 1 year ago
Thanks a lot for your work to better fine-tune BB2. It is interesting to see the extra improvement you achieved on the main tasks in BB2. I don't have a firm answer for the question here, other than the hyper parameters you used ended up helping the model to fine-tune better.
The released BB2 model is also fine-tuned on safety datasets to incorporate baked-in safety measures. Fine-tuning strictly on MSC and WoI will bring it more towards that distribution
This issue has not had activity in 30 days. Please feel free to reopen if you have more issues. You may apply the "never-stale" tag to prevent this from happening.
I trying to further fine-tune the given BB2_3B model on MSC + WOI datasets. I'm expect that the PPL scores on validation set won't change much after the fine-tuning. Surprisingly, I got a big PPL improvement on CONVAI2 task while the PPL scores for other tasks remain the same. . I realized that the PPL score of the original BB2 on CONVAI2 task is really bad compared to other tasks. Here is the evaluation results of the original BB2 model on validation set:
Here is the evaluation results after fine-tuning for 3000 steps (total: 24000 samples with 4000 samples/task)
As you can see, there is a big PPL improvement in msc:Session1Self task (CONVAI2) . I not really sure what happening there. Maybe the original BB2 model is not trained with CONVAI-2 samples? Or the BB2 model may not converge yet when the training end? . Here is my fine-tuning command: