mozilla / firefox-translations-training

Training pipelines for Firefox Translations neural machine translation models
https://mozilla.github.io/firefox-translations-training/
Mozilla Public License 2.0
145 stars 31 forks source link

Add a pre-release test for models #854

Open gregtatum opened 1 week ago

gregtatum commented 1 week ago

We had an incident where the uk-en model broke in Nightly. We should automate tests before release so that it catches issues like. We should run each release channel of Firefox and confirm end to end that the release works. I think the addon had something like this as well. We'll need something that can run browser chrome tests. Maybe it's appropriate to just check out Firefox and run a mochitest there. Or we can look at some other test runner that can do the tests.

Yes, I just confirmed that we still had a uk-en 1.0a1 model that was utilizing the split-vocab format. At the time I published the 1.1 version model, I didn't see in the list to remove because it was hidden behind the "load more" option in the Remote Settings UI.

For clarity:

The only experience that was broken should be translating from Ukrainian into another language. This experience was broken only on Nightly or local builds.

The removal of the alpha-version model is now up for review. Any of evgeny gregtatum or marco should be able to approve the changes, which will fix the issue.

iam_tj thanks for reporting this. It should be fixed and rolled out to Nightly users as soon as the removal is merged.

If you need it right away, you may want to manually re-sync from Remote Settings to ensure you eagerly receive the update. I can help with that if needed.