translation builds failing

[jbphet]

The last several translation builds have failed. Here is an example of one of the emails that was received:

Build failure: Error: ENOENT: no such file or directory, scandir '../gravity-and-orbits/build/phet'. Sim = gravity-and-orbits Version = 1.1.2 Locales = en_CA Shas = {"comment":"# gravity-and-orbits 1.1.2 Sun Jan 28 2018 02:17:39 GMT-0700 (MST)","assert":{"sha":"a707328cc11048787a807714496af23e0bd25be1","branch":"HEAD"},"axon":{"sha":"03e252e3f7252c9853e6080ab9b2cf5369f50580","branch":"HEAD"},"brand":{"sha":"517282e9244ae8eb2b195f1927aa02e6f7255805","branch":"HEAD"},"chipper":{"sha":"eb7cbe89a3b2a222163aac5fbae1bbc77a25aa59","branch":"HEAD"},"dot":{"sha":"9b3b950fd8f8a5c1efcef537ea066db7b23173d6","branch":"HEAD"},"gravity-and-orbits":{"sha":"988011594024fc5ba63e6466dfe890e9d5ba704a","branch":"HEAD"},"joist":{"sha":"9e4a5177282366f045a970bc4f4d93be67b7680b","branch":"HEAD"},"kite":{"sha":"f4c67cff6e24693f66b58fcea16069be3dbd8e88","branch":"HEAD"},"phet-core":{"sha":"c5c6c2a8273b9bd3f3bcb4ddfbcf24058b58954a","branch":"HEAD"},"phetcommon":{"sha":"573803951eef9249c977b73790d41283d864fe8a","branch":"HEAD"},"query-string-machine":{"sha":"d8a4ff18a12aff003e4896213c1dfed5a79c1a30","branch":"HEAD"},"scenery":{"sha":"3ce1f9825d4d02971e503b8b7d617ed44462ec65","branch":"HEAD"},"scenery-phet":{"sha":"9233aff82ac8e9366d444e72d40603e59a261493","branch":"HEAD"},"sherpa":{"sha":"985665662c9862d5b38583a715896029b46feb44","branch":"HEAD"},"sun":{"sha":"9c5de75906c0fa05d8535b83d394bc41374f5851","branch":"HEAD"},"tandem":{"sha":"4ac1125441b04a890088f9704314879aba5dead0","branch":"HEAD"}}

I discussed this with @mattpen, and he said that the build server appears to be looking for the directory '../gravity-and-orbits/build/phet and not finding it, which indicates that the sim is using a pre-2.0 version of chipper, but the build server thinks its a 2.0+ version. I looked at the chipper version for this build, and the package.json file contains the string "version": "0.0.0", which should be correct. Not sure what's going on here, but I think it means translations won't work until this is resolved.

[jbphet]

I've taken the translation utility off line over the weekend, and @mattpen and I set up a time to work on this on Monday Feb 5 2018.

[jbphet]

For reference, there were 7 email notifications of build failures that brought this to our attention. In the rosetta log, the first of the build requests was issued at Feb 02 2018 05:49:18 MST, and the last at Feb 02 2018 06:34:06 MST. There were no other build requests in the log after that time, and I disabled rosetta at around 5:30 PM (17:30) that same day.

[jbphet]

In working with @mattpen, it looks like there was a failure of the requireJS build for the first failure, and then some unhandled promises. We recalled some changes to promise handling in the build server, and @mattpen checked, and it looks like those changes were not deployed to perennial. Perennial has now been updated.

This should improve the accuracy of the error message that we are receiving, but doesn't address the root problem of why the RequireJS build failed.

[jonathanolson]

It looks like we may need an automated maintenance release to fix the locales in localeInfo on sims chipper versions. @ariel-phet, it would be generally good if I could upgrade the automated maintenance release process before that is done.

[jbphet]

@mattpen, @jonathanolson, and I have pieced together what was going on here. A translator mistakenly submitted translations for the locale en_CA for two simulations, Gravity and Orbits (GAO) and States of Matter (SOM). This locale was added fairly recently to handle a special situation, and I only did maintenance releases for the sims for which it was requested. GAO and SOM were not in that batch, so the locale is not supported by the live version of those sims. Rosetta doesn't know this, and allowed the strings files to be submitted to babel, the strings repo. The presence of these strings files causes the build to break entirely for these sims with an "unsupported locale" error. The user went on to try the process again using what I believe is the correct locale for this user, which is or (Oriya). These failed because of the previously described problem.

I've removed these two string files and tested locally, and the builds worked, so this should fix the immediate issue.

Long term, we should perform a maintenance release on all simulations to add support for this locale. Some sims have it already, but another locale has also been added, and this problem exposes a vulnerability in the translation+build system that should be corrected. Assigning to @jonathanolson to comment on the maintenance release (he and I discussed this, and he had some thoughts on how to do this in light of the recent chipper-2.0 work). After that, this should be assigned to @ariel-phet for prioritization or marked for discussion at dev meeting.

[jbphet]

I just noticed that @jonathanolson commented while I was updating the issue, so assigning to @ariel-phet for prioritization.

[jbphet]

@oliver-phet - can you let the translator who triggered this problem know that he needs to be careful about selecting the correct locale? The locale that he used later was or, which may well be correct. If so, he will need to re-submit those.

[jbphet]

For reference, here are the issues relating to somewhat recent addition of support for new locales:

https://github.com/phetsims/rosetta/issues/149 https://github.com/phetsims/rosetta/issues/158

[oliver-phet]

@jbphet Translator notified.

[ariel-phet]

Marking this for dev meeting as I would like to discuss with several parties. Generally I am leaning towards waiting a few weeks (I would like to get a few things polished off before we have @jonathanolson work on the automated MR process)

[ariel-phet]

@jonathanolson says this would be a good simple test of the maintenance release process once that is upgraded