mozilla / firefox-translations-training

Training pipelines for Firefox Translations neural machine translation models
https://mozilla.github.io/firefox-translations-training/
Mozilla Public License 2.0
145 stars 31 forks source link

Optimize alignments #703

Closed eu9ene closed 3 months ago

eu9ene commented 3 months ago

Known issues:

fixes #507 fixes #663

eu9ene commented 3 months ago

The tests will fail due to https://github.com/mozilla/firefox-translations-training/issues/689

eu9ene commented 3 months ago

It turned out to be hard to restart only the alignments-original, so I added an extra task to recalculate the priors for the student alignments (in a separate branch). I'm testing it here for en-uk https://firefox-ci-tc.services.mozilla.com/tasks/groups/eZKkxqHISTCDwrsylAZZvA. This way we won't need to rerun things multiple times.