search

phetsims / rosetta

PhET's Simulation Translation Utility
MIT License
3 stars 1 forks source link

Investigate behavior of updating translation stats #401

Open liammulh opened 1 year ago

liammulh commented 1 year ago

Right now the behavior seems incorrect. @KatieWoe pointed this out to me in her testing of https://github.com/phetsims/qa/issues/931.

I think it would be a good idea to break this problem down into smaller problems:

When does the "(pending update)" badge appear?

  1. If you translate a sim-specific string, does the "(pending update)" badge appear next to the sim?
  2. If you translate a shared string, does the "(pending update)" badge appear next to the sim and the other sims that share the string?
  3. If you translate a common string, do all sims that have the common string have a "(pending update)" badge?

How long does it take for the stats to update?

At this point, the only bottleneck for having perfectly up-to-date translation stats is the build server. We can get fresh sim metadata (since we bust the cache) and we can get fresh GitHub data (since we are now using Octokit).

  1. If you translate a sim-specific string, how long does it take for the stats to update?
  2. If you translate a shared string, how long does it take for the stats to update?
  3. If you translate a common string, how long does it take for the stats to update?
liammulh commented 1 year ago

We now have a flush report object route and GUI (under the admin panel). If we discover incorrect stats, we should flush the incorrect report object(s). This route flushes the stats for a locale/sim combination from the report object cache in Rosetta's memory.

Long-term, it would be nice to pin down exactly what might be going wrong with the stats updates.

If you translate a common string, do all sims that have the common string have a "(pending update)" badge?

No, if you translate a common string, the sim that was translated will have a "(pending update)" badge, and (I think) the sims that share strings with it will have a "(pending update)" badge. The other sims that share the common string will not get a "(pending update)" badge.

@jbphet and I discussed flushing the whole translation report for a locale if a common string is translated in that locale. There are two big drawbacks to this approach:

  1. Each translation report requires ~800 GitHub API requests, and we only have 5000 per hour. See https://github.com/phetsims/rosetta/issues/410 for more info.
  2. If we flush the report object for a locale and then the translator immediately reloads the translation report for that locale, they will see up-to-date translation statistics, but the tables (published versus unpublished sims) might be incorrect because the build server hasn't had enough time to build the translations.

Occasionally restarting Rosetta to deploy new patches and features has been (we think) working reasonably well so far. If @jbphet is going to put Rosetta on the back burner, it might make sense to have a cron-job that restarts Rosetta every once in a while? Or perhaps it would make sense to provide a button that reloads the stats for a given sim? Kind of like the "pull" button on phettest? I don't think we'd want to provide the users with a button that reloads a whole translation report because they might click it a lot, and we are limited on GitHub API requests.