phetsims / chipper

Tools for developing and building PhET interactive simulations.
MIT License
11 stars 14 forks source link

Find a better way to get translated screen names for the website #1367

Closed jonathanolson closed 1 year ago

jonathanolson commented 1 year ago

It sounds like it's taking 7 minutes to parse out the screen names right now (but running the simulation for every locale). This seems inefficient, perhaps we can generate these during the simulation build quickly (especially if sims report out what their string keys are for the screens).

mattpen commented 1 year ago

especially if sims report out what their string keys are for the screens

Even if this is all we could provide to the build-server, it would be pretty simple to fetch these from babel - much more efficient than running puppeteer to scrape the screen names from joist at runtime.


Also, if this proves difficult, one other option is for the website to do this asyncrounously whenever it generates the other file related content for the metadata (currently scheduled once/day). This would accomplish the goal of speeding the builds up anyway.

jonathanolson commented 1 year ago

This would allow us to NOT run puppeteer to grab the screen names out of sims, AND would be fairly feasible to patch for older simulations AND master. THIS would allow us to remove puppeteer usage from build-server, which would make it much more robust and faster (although phet-io brand builds will still run puppeteer).

This would knock off at least 8 hours from a full production deployment of all release branches.

I'll need to talk to other devs about this, but here's my proposal:

We include an array of screen name string keys in the package.json, e.g. in area-model-algebra's package.json:

    "screenNameKeys": [
      "AREA_MODEL_COMMON/screen.explore",
      "AREA_MODEL_COMMON/screen.generic",
      "AREA_MODEL_COMMON/screen.variables",
      "AREA_MODEL_COMMON/screen.game"
    ]

This would only be done for multi-screen sims (would not exist for single-screen sims).

This can be generated for most older sims by running puppeteer and filtering keys with something like:

const likelyKeys = Object.keys( phet.chipper.strings.en ).filter( str => str.includes( 'creen' ) && !str.includes( 'JOIST' ) && !str.includes( 'SCENERY_PHET' ) );
_.find( likelyKeys, key => phet.chipper.strings.en[ key ] === phet.joist.sim.screens[ 1 ].nameProperty.value )

Additionally, on master, we can add code to make sure this doesn't get out-of-sync. It's possible to tag the stringProperties with a stringKey field (with minor modifications to getStringModule and LocalizedString), and then we can add assertions to make sure they match up.

jonathanolson commented 1 year ago

I'll also add assertions in Screen to make sure that the key provided is in the package.json, so it doesn't get out-of-sync.

jonathanolson commented 1 year ago

We have some cases where non-translated screen names are in published sims, e.g. in circuit-construction-kit-black-box-study 1.1-phetio phet-io:

This looks like a phet-io case, @mattpen can we ignore phet-io-only branches?

jonathanolson commented 1 year ago

Script used for maintenance:


const getActiveRunnables = require( './js/common/getActiveRunnables' );
const puppeteerEvaluate = require( './js/common/puppeteerEvaluate' );
const loadJSON = require( './js/common/loadJSON' );
const writeJSON = require( './js/common/writeJSON' );
const Maintenance = require( './js/common/Maintenance' );
const gitCheckout = require( './js/common/gitCheckout' );
const gitAdd = require( './js/common/gitAdd' );
const gitCommit = require( './js/common/gitCommit' );
const gitPush = require( './js/common/gitPush' );
const puppeteer = require( 'puppeteer' );

const getKeysFromURL = async ( url, browser ) => {
  return puppeteerEvaluate( url, () => {
    const nameFromScreen = screen => {
      return screen.nameProperty ? screen.nameProperty.value : screen.name;
    };
    const isHomeScreen = screen => {
      return nameFromScreen( screen )?.includes( 'Home' );
    };
    const screens = phet.joist.sim.screens.filter( screen => !isHomeScreen( screen ) );
    const findPotentialKeys = string => Object.keys( phet.chipper.strings.en ).filter( key => phet.chipper.strings.en[ key ] === string );

    // Only do this for multi-screen sims
    if ( screens.length < 2 ) {
      return [];
    }

    return screens.map( screen => {
      const potentialKeys = findPotentialKeys( nameFromScreen( screen ) );
      if ( potentialKeys.length === 0 ) {
        return `Cannot find any matching strings for ${nameFromScreen( screen )}`;
      }
      if ( potentialKeys.length === 1 ) {
        return potentialKeys[ 0 ];
      }
      const moreLikelyKeys = potentialKeys.filter( key => key.includes( 'creen' ) && !key.includes( 'JOIST' ) && !key.includes( 'SCENERY_PHET' ) );
      if ( moreLikelyKeys.length === 1 ) {
        return moreLikelyKeys[ 0 ];
      }
      else {
        return `Cannot disambiguate keys for ${nameFromScreen( screen )}: ${potentialKeys}`;
      }
    } );
  } );
};

( async () => {

  const browser = await puppeteer.launch( {
    args: [
      '--disable-gpu'
    ]
  } );

  // Force cache break for now
  const releaseBranches = await Maintenance.getMaintenanceBranches( () => true, true, true );

  for ( const releaseBranch of releaseBranches ) {
    console.log( releaseBranch.toString() );
    const builtURL = releaseBranch.brands.includes( 'phet' ) ? await releaseBranch.getLocalPhetBuiltHTMLPath() : `${await releaseBranch.getLocalPhetIOBuiltHTMLPath()}?${await releaseBranch.getPhetioStandaloneQueryParameter()}`;
    const fullURL = `http://localhost/.maintenance/${releaseBranch.repo}-${releaseBranch.branch}/${releaseBranch.repo}/${builtURL}`;
    console.log( fullURL );
    const keys = await getKeysFromURL( fullURL, browser );
    console.log( keys );

    if ( typeof keys !== 'string' && keys.length ) {
      await gitCheckout( releaseBranch.repo, releaseBranch.branch );

      const packageFile = `../${releaseBranch.repo}/package.json`;
      const packageObject = await loadJSON( packageFile );

      packageObject.phet.screenNameKeys = keys;

      await writeJSON( packageFile, packageObject );
      await gitAdd( releaseBranch.repo, 'package.json' );
      await gitCommit( releaseBranch.repo, 'Adding screenNameKeys, see https://github.com/phetsims/chipper/issues/1367' );
      await gitPush( releaseBranch.repo, releaseBranch.branch );

      await gitCheckout( releaseBranch.repo, 'master' );
    }
  }

  await browser.close();
} )();
jonathanolson commented 1 year ago

RCs, still queued up:

acid-base-solutions 1.2 (https://github.com/phetsims/chipper/issues/1367)

area-builder 1.1 (https://github.com/phetsims/chipper/issues/1367)

area-model-algebra 1.2 (https://github.com/phetsims/chipper/issues/1367)

area-model-decimals 1.2 (https://github.com/phetsims/chipper/issues/1367)

area-model-introduction 1.2 (https://github.com/phetsims/chipper/issues/1367)

area-model-multiplication 1.2 (https://github.com/phetsims/chipper/issues/1367)

arithmetic 1.0 (https://github.com/phetsims/chipper/issues/1367)

atomic-interactions 1.2 (https://github.com/phetsims/chipper/issues/1367)

balancing-act 1.1 (https://github.com/phetsims/chipper/issues/1367)

balancing-chemical-equations 1.2 (https://github.com/phetsims/chipper/issues/1367)

balloons-and-static-electricity 1.3-phetio (https://github.com/phetsims/chipper/issues/1367)

balloons-and-static-electricity 1.5 (https://github.com/phetsims/chipper/issues/1367)

beers-law-lab 1.7 (https://github.com/phetsims/chipper/issues/1367)

bending-light 1.1 (https://github.com/phetsims/chipper/issues/1367)

blackbody-spectrum 1.0 (https://github.com/phetsims/chipper/issues/1367)

build-a-fraction 1.0 (https://github.com/phetsims/chipper/issues/1367)

build-a-molecule 1.0 (https://github.com/phetsims/chipper/issues/1367)

build-a-nucleus 1.0 (https://github.com/phetsims/chipper/issues/1367)

build-an-atom 1.5-phetio (https://github.com/phetsims/chipper/issues/1367)

build-an-atom 1.6 (https://github.com/phetsims/chipper/issues/1367)

calculus-grapher 1.0 (https://github.com/phetsims/chipper/issues/1367)

capacitor-lab-basics 1.6 (https://github.com/phetsims/chipper/issues/1367)

center-and-variability 1.0 (https://github.com/phetsims/chipper/issues/1367)

charges-and-fields 1.0 (https://github.com/phetsims/chipper/issues/1367)

charges-and-fields 1.0-phetio (https://github.com/phetsims/chipper/issues/1367)

circuit-construction-kit-ac 1.0 (https://github.com/phetsims/chipper/issues/1367)

circuit-construction-kit-ac-virtual-lab 1.0 (https://github.com/phetsims/chipper/issues/1367)

circuit-construction-kit-dc 1.3 (https://github.com/phetsims/chipper/issues/1367)

circuit-construction-kit-dc-virtual-lab 1.3 (https://github.com/phetsims/chipper/issues/1367)

collision-lab 1.1 (https://github.com/phetsims/chipper/issues/1367)

color-vision 1.1 (https://github.com/phetsims/chipper/issues/1367)

color-vision 1.2-phetio (https://github.com/phetsims/chipper/issues/1367)

concentration 1.5-phetio (https://github.com/phetsims/chipper/issues/1367)

concentration 1.7 (https://github.com/phetsims/chipper/issues/1367)

coulombs-law 1.0 (https://github.com/phetsims/chipper/issues/1367)

curve-fitting 1.0 (https://github.com/phetsims/chipper/issues/1367)

density 1.0 (https://github.com/phetsims/chipper/issues/1367)

diffusion 1.0 (https://github.com/phetsims/chipper/issues/1367)

energy-forms-and-changes 1.4 (https://github.com/phetsims/chipper/issues/1367)

energy-skate-park 1.2 (https://github.com/phetsims/chipper/issues/1367)

energy-skate-park-basics 1.1 (https://github.com/phetsims/chipper/issues/1367)

energy-skate-park-basics 1.3-phetio (https://github.com/phetsims/chipper/issues/1367)

equality-explorer 1.1 (https://github.com/phetsims/chipper/issues/1367)

equality-explorer-basics 1.0 (https://github.com/phetsims/chipper/issues/1367)

equality-explorer-two-variables 1.0 (https://github.com/phetsims/chipper/issues/1367)

expression-exchange 1.1 (https://github.com/phetsims/chipper/issues/1367)

faradays-law 1.3-phetio (https://github.com/phetsims/chipper/issues/1367)

faradays-law 1.4 (https://github.com/phetsims/chipper/issues/1367)

forces-and-motion-basics 2.1-phetio (https://github.com/phetsims/chipper/issues/1367)

forces-and-motion-basics 2.3 (https://github.com/phetsims/chipper/issues/1367)

fourier-making-waves 1.0 (https://github.com/phetsims/chipper/issues/1367)

fraction-matcher 1.2 (https://github.com/phetsims/chipper/issues/1367)

fractions-equality 1.1 (https://github.com/phetsims/chipper/issues/1367)

fractions-intro 1.0 (https://github.com/phetsims/chipper/issues/1367)

fractions-mixed-numbers 1.0 (https://github.com/phetsims/chipper/issues/1367)

friction 1.6 (https://github.com/phetsims/chipper/issues/1367)

function-builder 1.2 (https://github.com/phetsims/chipper/issues/1367)

function-builder-basics 1.2 (https://github.com/phetsims/chipper/issues/1367)

gas-properties 1.0 (https://github.com/phetsims/chipper/issues/1367)

gases-intro 1.0 (https://github.com/phetsims/chipper/issues/1367)

gene-expression-essentials 1.0 (https://github.com/phetsims/chipper/issues/1367)

geometric-optics 1.1 (https://github.com/phetsims/chipper/issues/1367)

geometric-optics-basics 1.2 (https://github.com/phetsims/chipper/issues/1367)

graphing-lines 1.3 (https://github.com/phetsims/chipper/issues/1367)

graphing-quadratics 1.2 (https://github.com/phetsims/chipper/issues/1367)

graphing-slope-intercept 1.1 (https://github.com/phetsims/chipper/issues/1367)

gravity-and-orbits 1.5 (https://github.com/phetsims/chipper/issues/1367)

gravity-and-orbits 1.6 (https://github.com/phetsims/chipper/issues/1367)

gravity-force-lab 2.2 (https://github.com/phetsims/chipper/issues/1367)

gravity-force-lab-basics 1.1 (https://github.com/phetsims/chipper/issues/1367)

greenhouse-effect 1.0 (https://github.com/phetsims/chipper/issues/1367)

greenhouse-effect 1.1 (https://github.com/phetsims/chipper/issues/1367)

hookes-law 1.0 (https://github.com/phetsims/chipper/issues/1367)

isotopes-and-atomic-mass 1.1 (https://github.com/phetsims/chipper/issues/1367)

john-travoltage 1.4-phetio (https://github.com/phetsims/chipper/issues/1367)

john-travoltage 1.6 (https://github.com/phetsims/chipper/issues/1367)

least-squares-regression 1.1 (https://github.com/phetsims/chipper/issues/1367)

make-a-ten 1.0 (https://github.com/phetsims/chipper/issues/1367)

masses-and-springs 1.0 (https://github.com/phetsims/chipper/issues/1367)

masses-and-springs-basics 1.0 (https://github.com/phetsims/chipper/issues/1367)

mean-share-and-balance 1.0 (https://github.com/phetsims/chipper/issues/1367)

molarity 1.4 (https://github.com/phetsims/chipper/issues/1367)

molarity 1.5 (https://github.com/phetsims/chipper/issues/1367)

molecule-polarity 1.2 (https://github.com/phetsims/chipper/issues/1367)

molecule-shapes 1.6 (https://github.com/phetsims/chipper/issues/1367)

molecule-shapes-basics 1.6 (https://github.com/phetsims/chipper/issues/1367)

molecules-and-light 1.3-phetio (https://github.com/phetsims/chipper/issues/1367)

molecules-and-light 1.5 (https://github.com/phetsims/chipper/issues/1367)

my-solar-system 1.0 (https://github.com/phetsims/chipper/issues/1367)

natural-selection 1.2 (https://github.com/phetsims/chipper/issues/1367)

natural-selection 1.3 (https://github.com/phetsims/chipper/issues/1367)

natural-selection 1.4 (https://github.com/phetsims/chipper/issues/1367)

neuron 1.1 (https://github.com/phetsims/chipper/issues/1367)

normal-modes 1.0 (https://github.com/phetsims/chipper/issues/1367)

number-compare 1.0 (https://github.com/phetsims/chipper/issues/1367)

number-line-distance 1.0 (https://github.com/phetsims/chipper/issues/1367)

number-line-integers 1.1 (https://github.com/phetsims/chipper/issues/1367)

number-line-operations 1.0 (https://github.com/phetsims/chipper/issues/1367)

number-play 1.1 (https://github.com/phetsims/chipper/issues/1367)

ohms-law 1.4 (https://github.com/phetsims/chipper/issues/1367)

pendulum-lab 1.0 (https://github.com/phetsims/chipper/issues/1367)

ph-scale 1.5 (https://github.com/phetsims/chipper/issues/1367)

ph-scale 1.6 (https://github.com/phetsims/chipper/issues/1367)

ph-scale-basics 1.5 (https://github.com/phetsims/chipper/issues/1367)

ph-scale-basics 1.6 (https://github.com/phetsims/chipper/issues/1367)

phet-io-test-sim 2.12 (https://github.com/phetsims/chipper/issues/1367)

plinko-probability 1.1 (https://github.com/phetsims/chipper/issues/1367)

projectile-motion 1.0 (https://github.com/phetsims/chipper/issues/1367)

proportion-playground 1.0 (https://github.com/phetsims/chipper/issues/1367)

ratio-and-proportion 1.2 (https://github.com/phetsims/chipper/issues/1367)

reactants-products-and-leftovers 1.2 (https://github.com/phetsims/chipper/issues/1367)

resistance-in-a-wire 1.3-phetio (https://github.com/phetsims/chipper/issues/1367)

resistance-in-a-wire 1.6 (https://github.com/phetsims/chipper/issues/1367)

rutherford-scattering 1.1 (https://github.com/phetsims/chipper/issues/1367)

states-of-matter 1.2 (https://github.com/phetsims/chipper/issues/1367)

states-of-matter-basics 1.2 (https://github.com/phetsims/chipper/issues/1367)

trig-tour 1.0 (https://github.com/phetsims/chipper/issues/1367)

under-pressure 1.1 (https://github.com/phetsims/chipper/issues/1367)

unit-rates 1.0 (https://github.com/phetsims/chipper/issues/1367)

vector-addition 1.0 (https://github.com/phetsims/chipper/issues/1367)

vector-addition-equations 1.0 (https://github.com/phetsims/chipper/issues/1367)

wave-interference 2.0 (https://github.com/phetsims/chipper/issues/1367)

wave-on-a-string 1.1 (https://github.com/phetsims/chipper/issues/1367)

waves-intro 1.1 (https://github.com/phetsims/chipper/issues/1367)

jonathanolson commented 1 year ago

I'm going to do a scan of commits to make sure nothing snuck in:

jonathanolson commented 1 year ago

Production links, I'll test:

acid-base-solutions 1.2 ()

area-builder 1.1 ()

area-model-algebra 1.2 ()

area-model-decimals 1.2 ()

area-model-introduction 1.2 ()

area-model-multiplication 1.2 ()

arithmetic 1.0 ()

atomic-interactions 1.2 ()

balancing-act 1.1 ()

balancing-chemical-equations 1.2 ()

balloons-and-static-electricity 1.3-phetio ()

balloons-and-static-electricity 1.5 ()

beers-law-lab 1.7 ()

bending-light 1.1 ()

blackbody-spectrum 1.0 ()

build-a-fraction 1.0 ()

build-a-molecule 1.0 ()

build-a-nucleus 1.0 ()

build-an-atom 1.5-phetio ()

build-an-atom 1.6 ()

calculus-grapher 1.0 ()

capacitor-lab-basics 1.6 ()

center-and-variability 1.0 ()

charges-and-fields 1.0 ()

charges-and-fields 1.0-phetio ()

circuit-construction-kit-ac 1.0 ()

circuit-construction-kit-ac-virtual-lab 1.0 ()

circuit-construction-kit-dc 1.3 ()

circuit-construction-kit-dc-virtual-lab 1.3 ()

collision-lab 1.1 ()

color-vision 1.1 ()

color-vision 1.2-phetio ()

concentration 1.5-phetio ()

concentration 1.7 ()

coulombs-law 1.0 ()

curve-fitting 1.0 ()

density 1.0 ()

diffusion 1.0 ()

energy-forms-and-changes 1.4 ()

energy-skate-park 1.2 ()

energy-skate-park-basics 1.1 ()

energy-skate-park-basics 1.3-phetio ()

equality-explorer 1.1 ()

equality-explorer-basics 1.0 ()

equality-explorer-two-variables 1.0 ()

expression-exchange 1.1 ()

faradays-law 1.3-phetio ()

faradays-law 1.4 ()

forces-and-motion-basics 2.1-phetio ()

forces-and-motion-basics 2.3 ()

fourier-making-waves 1.0 ()

fraction-matcher 1.2 ()

fractions-equality 1.1 ()

fractions-intro 1.0 ()

fractions-mixed-numbers 1.0 ()

friction 1.6 ()

function-builder 1.2 ()

function-builder-basics 1.2 ()

gas-properties 1.0 ()

gases-intro 1.0 ()

gene-expression-essentials 1.0 ()

geometric-optics 1.1 ()

geometric-optics-basics 1.2 ()

graphing-lines 1.3 ()

graphing-quadratics 1.2 ()

graphing-slope-intercept 1.1 ()

gravity-and-orbits 1.5 ()

gravity-and-orbits 1.6 ()

gravity-force-lab 2.2 ()

gravity-force-lab-basics 1.1 ()

greenhouse-effect 1.0 ()

greenhouse-effect 1.1 ()

hookes-law 1.0 ()

isotopes-and-atomic-mass 1.1 ()

john-travoltage 1.4-phetio ()

john-travoltage 1.6 ()

least-squares-regression 1.1 ()

make-a-ten 1.0 ()

masses-and-springs 1.0 ()

masses-and-springs-basics 1.0 ()

mean-share-and-balance 1.0 ()

molarity 1.4 ()

molarity 1.5 ()

molecule-polarity 1.2 ()

molecule-shapes 1.6 ()

molecule-shapes-basics 1.6 ()

molecules-and-light 1.3-phetio ()

molecules-and-light 1.5 ()

my-solar-system 1.0 ()

natural-selection 1.2 ()

natural-selection 1.3 ()

natural-selection 1.4 ()

neuron 1.1 ()

normal-modes 1.0 ()

number-compare 1.0 ()

number-line-distance 1.0 ()

number-line-integers 1.1 ()

number-line-operations 1.0 ()

number-play 1.1 ()

ohms-law 1.4 ()

pendulum-lab 1.0 ()

ph-scale 1.5 ()

ph-scale 1.6 ()

ph-scale-basics 1.5 ()

ph-scale-basics 1.6 ()

phet-io-test-sim 2.12 ()

plinko-probability 1.1 ()

projectile-motion 1.0 ()

proportion-playground 1.0 ()

ratio-and-proportion 1.2 ()

reactants-products-and-leftovers 1.2 ()

resistance-in-a-wire 1.3-phetio ()

resistance-in-a-wire 1.6 ()

rutherford-scattering 1.1 ()

states-of-matter 1.2 ()

states-of-matter-basics 1.2 ()

trig-tour 1.0 ()

under-pressure 1.1 ()

unit-rates 1.0 ()

vector-addition 1.0 ()

vector-addition-equations 1.0 ()

wave-interference 2.0 ()

wave-on-a-string 1.1 ()

waves-intro 1.1 ()

jonathanolson commented 1 year ago

Verified that all production releases completed successfully.

mattpen commented 1 year ago

@jonathanolson and I successfully adapted the build-server today to use the translated screen names based on the sim's package.json and the relevant babel files. I confirmed that this is working as expected with a production deploy of example-sim v1.2. This reduced the amount of time needed for screen names from 3-5 minutes to less than 1 second! @jonathanolson - is there anything else to do here?

jonathanolson commented 1 year ago

Nothing else to do, closing!