search

mozilla / translations

The code, training pipeline, and models that power Firefox Translations
https://mozilla.github.io/translations/
Mozilla Public License 2.0
154 stars 33 forks source link

One of the teachers for el-en diverged #723

Closed eu9ene closed 3 months ago

eu9ene commented 3 months ago

The second one looks fine.

https://wandb.ai/moz-translations/el-en/workspace

Can it be just a local minima? We use random seeds so restart doesn't help. Maybe changing the learning rate for this language can help here.

Screenshot 2024-07-08 at 11 05 01 AM
eu9ene commented 3 months ago

Reducing the learning rate to 0.0002 helped