This is a meta issue to brainstorm ideas on how to make our final automatic quality evaluation procedure more robust and suitable for decision-making. This currently happens in firefox-translations-models repo but will be migrated to the main repo and W&B.
This is a meta issue to brainstorm ideas on how to make our final automatic quality evaluation procedure more robust and suitable for decision-making. This currently happens in firefox-translations-models repo but will be migrated to the main repo and W&B.
See this doc.
Some ideas:
Recommendations from the paper Navigating the Metrics Maze: Reconciling Score Magnitudes and Accuracies:
We can add a task list here after migrating [firefox-translations-models].