However, I noticed a load and save issue when fine-tuning models.
Whenever I finetuned a model and reloaded it, it achieved terrible evaluation scores (worse than without finetuning, even on the training data).
After tinkering for a while, I noticed that this even happens if I set the learning rate to zero. Thus, I assumed the model weights had a load/save issue.
Commenting out contriever.py:123-126, however, solved the issue for me. Otherwise, it seems the model weights of a checkpoint are never mapped to the newly instantiated model.
Am I on the right track? Are there any drawbacks to commenting out the lines? (I assume there is an intention behind them I do not fully grasp).
P.S:
pretty sure finetuning_data.py:47 should refer to self.negative_hard_min_idx not self.hard_negative_min_idx.
Hi,
First of all, thank you for the great work!
However, I noticed a load and save issue when fine-tuning models. Whenever I finetuned a model and reloaded it, it achieved terrible evaluation scores (worse than without finetuning, even on the training data).
After tinkering for a while, I noticed that this even happens if I set the learning rate to zero. Thus, I assumed the model weights had a load/save issue.
Commenting out
contriever.py:123-126
, however, solved the issue for me. Otherwise, it seems the model weights of a checkpoint are never mapped to the newly instantiated model.Am I on the right track? Are there any drawbacks to commenting out the lines? (I assume there is an intention behind them I do not fully grasp).
P.S:
pretty sure
finetuning_data.py:47
should refer toself.negative_hard_min_idx
notself.hard_negative_min_idx
.Best regards,
Tim