version id scheme

[pixelzoom]

The version id for PhET sims has (imo) gotten a bit confused with the introduction of PhET-iO. The 'phetio' brand identifier has been tacked on in a non-general way, and I have yet to see anyone provide a coherent/comprehensive description of the version id's syntax and semantics (getVersionForBrand.js included).

Current version id scheme

Here is the current format for public, dev and RC versions respectively, when brand is 'phet':

1.1.0 1.1.0-dev.1 1.1.0-rc.1

So far, so good. Now here is the current format when brand is 'phetio':

1.1.0-phetio 1.1.0-phetiodev.1 1.1.0-phetiorc.1

In the public version case, the build process adds a '-phetio' prefix. In the dev and RC cases, the build process reads the version id from package.json and replaces 'dev' and 'rc' with 'phetiodev' and 'phetiorc' respectively.

A few problems with this scheme:

(1) It modifies what the developer puts in package.json. (2) It modifies test and public versions differently. (3) It combines the "test type" and "brand" info, making it more difficult to parse. (4) It is not general enough to support future test types or brands. (5) It's difficult to write a "general form" description.

I'm proposing that we revised the version id scheme, as described below.

Proposed version id scheme

The version id scheme will consists of 3 chunks:

number[-test][-brand]

where:

• number describes the version number • test describes the test (absence indicates a public version that has been QA tested) •brand describes the brand (absence indicates 'phet' brand) • square brackets indicate an optional chunk • hypen is the separator between chunks

Expanding the chunks, we have:

major.minor.maintenance[-testType.testNumber][-brandName]

where:

• major = major number, sequential integer • minor = minor number, sequential integer, resets to 0 when major is changed • maintenance = maintenance number, sequential integer, resets to 0 when minor is changed • typeType = the type of test, currently 'dev' or 'rc' • testNumber = sequential integer indicating the test number • brandName = the brand name, e.g. 'phetio'. Absence indicates 'phet' brand.

The developer specifies the number[-test] portion of the version identifier in package.json. The build process add the [-brandName] portion, if applicable, to each supported brand that is built.

Examples:

1.5.0 = public version of 'phet' brand 1.5.0-phetio = public version of 'phetio' brand 1.5.0-x = public version of brand 'x', where 'x' is a client or some future brand 1.5.0-dev.1 = dev test for 'phet' brand 1.5.0-dev.1-phetio = dev test for 'phetio' brand 1.5.0-dev.1-x = dev test for brand 'x' 1.5.0-rc.1 = RC test for 'phet' brand 1.5.0-rc.1-phetio = RC test for 'phetio' brand 1.5.0-rc.1-x = RC test for brand 'x'

Another advantage of this scheme will become apparent if/when we publish all supported brands for a sim simultaneously. We would want all related files to live in the same directory, with version ids that are clearly similar. For example, for a sim with supported brands 'phet', 'phetio' and 'x':

public versions, all located in 1.0.0/ directory: 1.0.0 1.0.0-phetio 1.0.0-x

dev versions, all located in 1.0.0-dev.15/ directory: 1.0.0-dev.15 1.0.0-dev.15-phetio 1.0.0-dev.15-x

RC version, all located in 1.0.0-rc.1/ directory: 1.0.0-rc.1 1.0.0-rc.1-phetio 1.0.0-rc.1-x

Labeling for discussion at developer meeting.

EDIT (@samreid): fixed a typo

[pixelzoom]

Yep, fixed in https://github.com/phetsims/chipper/issues/560#issuecomment-349709083

[jonathanolson]

Changes required to change the brands (particularly phet-io to phetio, NOTE it's probably possible to choose some subset of this instead of all of it, but that removes most of the "benefits"):

• All code usages of 'phet-io would presumably be changed to 'phetio', similar for filenames, directories, imports (PHET_IO => PHETIO), comments, etc.
  • There is in the range of ~2400 matches across our codebase for phet-io.
  • This doesn't count all of the potential renaming in server directories, server configs, etc.
• Query parameters for brands would change (brand=phetio instead, potentially breaking change unless we handle backwards compatibility)
• Rename repositories (phet-io, phet-io-website, phet-io-*)
  • This might cause a lot of broken links.
  • Any time we'd need to build an older (phet-io) branded simulation, we'd need to remap dependencies.json to change the repository names (a type of change we've never had to do before, as we haven't renamed any repositories after they were used for deployments)
• What do we do with existing phet-io sims?
• Presumably we keep phet-io.colorado.edu (so links don't break), or are we recommending changing to phetio.colorado.edu (new certificate, routing, etc.)
• Do we need to keep the branding, or will PhET-iO be changing? Presumably it stays the same, and we live with the inconsistency.
• Probably tons of things on the phet-io side that I'm not aware of. @samreid or @zepumph, does this break backwards compatibility or cause any major issues?
• Chipper/perennial build tools (this is probably minimal cost, already working on it). May need to add patches for backwards compatibility
• Will require reworking and full testing on:
  • Phet-io website/Apache (phet-io is baked into a lot of the directory names and configuration)
  • Yotta (will need to handle backwards compatibility with brands from sim messages)
  • Bayes continuous testing (has a lot of phet-io references, naming and specific behavior)
  • Phetmarks
  • Phettest.colorado.edu

Benefits received from the renaming:

• File version names are slightly easier to do/parse.
• We would have less of a mix of names in code, since - isn't allowed in variable names (still would probably keep some things, like phet-io.colorado.edu)

Instead, I'd propose we would ONLY change the suffix on built filenames, so e.g.:

faradays-law/

  1.0.0-dev.1/
    phet/
      faradays-law_en-phet.html
      faradays-law_all-phet.html
      faradays-law_all-phet-debug.html
    phet-io/
      faradays-law_all-phetio.html
      faradays-law_all-phetio-debug.html

If that is unacceptable, could we consider a different "separator" so that we could keep the brand names the same, e.g.:

faradays-law/

  1.0.0-dev.1/
    phet/
      faradays-law_en_phet.html
      faradays-law_all_phet.html
      faradays-law_all_phet_debug.html
    phetio/
      faradays-law_all_phet-io.html
      faradays-law_all_phet-io_debug.html

Since we tend to allow dashes in names (but not underscores), using the underscore as a separator works well.

[zepumph]

Tagging for dev meeting tomorrow.

[samreid]

Previously, @kathy-phet has expressed a strong preference for keeping the hyphen where PhET-iO appears in URLs, filenames and other user-visible locations.

[zepumph]

So by that we should probably have:

build/
    phet-io/
      faradays-law_all{{?}}phet-io.html
      faradays-law_all{{?}}phet-io{{?}}debug.html

[zepumph]

I'm not sure if this relates totally, but we should also try to factor out these names to a single place in chipper/perennial so there isn't a proliferation of assuming that the brand name won't change:

[jonathanolson]

Also noting:

• What should versions be in the About dialog or anything that represents the running sim version? (Presumably it would include the brand always, even if it is -phet).
• Yotta was using the previous assumption that brands wouldn't use capital letters (among other things), so it will need to be upgraded anyways with whatever we decide.

[ariel-phet]

12/7/17 revisions to https://github.com/phetsims/chipper/issues/560#issuecomment-348304508 based on decision in https://github.com/phetsims/chipper/issues/560#issuecomment-349051862:

local and dev directory structure (assuming brands=[phet,phet-io,adapted-from-phet])

build/ (local)
faradays-law/1.0.0/ (dev)
    phet/
      faradays-law_en_phet.html
      faradays-law_zh_CN_phet.html (locale-specific file, built only if specified)
      faradays-law_all_phet.html
      faradays-law_all_phet_debug.html
    phet-io/
      faradays-law_all_phet-io.html
      faradays-law_all_phet-io_debug.html
    adapted-from-phet/
      faradays-law_en_adapted-from-phet.html
      faradays-law_zh_CN_adapted-from-phet.html (locale-specific file, built only if specified)
      faradays-law_all_adapted-from-phet.html
      faradays-law_all_adapted-from-phet_debug.html

  1.0.0-dev.1/ and 1.0.0-rc.1/ same as 1.0.0/ 
  (by default, locale-specific files are only built on production)

 // one-off is in the directory name
  1.0.1-dev.1-sonification/
    phet/
      faradays-law_en_phet_sonification.html
      faradays-law_all_phet_sonification.html
      faradays-law_all_phet_sonification_debug.html
    phet-io/
      faradays-law_all_phet-io_sonification.html
      faradays-law_all_phet-io_sonification_debug.html
    adapted-from-phet/
      faradays-law_all_adapted-from-phet_sonification.html
      faradays-law_all_adapted-from-phet_sonification_debug.html

PhET production directory structure

faradays-law/
  1.0.0/
      faradays-law_en.html
      faradays-law_zh_CN.html
      faradays-law_all.html

PhET-iO production directory structure

faradays-law/
  1.0.0/
      faradays-law_all_phet-io.html
      faradays-law_all_phet-io_debug.html

[pixelzoom]

12/7/17 dev meeting notes, changes reflected in https://github.com/phetsims/chipper/issues/560#issuecomment-350054169:

• Allow dashes in brand name • Keep "phet-io" and "adapted-from-phet" brand names • Changed to underscore ('_') separator • Continue to us underscore separator in locales (to separate language and country, e.g. "zh_CN") • Version string (About dialog) and anything named 'version' in sim (e.g. globals) will not include the brand. • TODO: Document this somewhere when it becomes reality. (see #636)

[mattpen]

Discussed with @jonathanolson. We think that the build-server should fail and return an error if adapted-from-phet is in the list of brands for a production deploy.

[jonathanolson]

Implemented in the build and dev deploy. Will be waiting on the build-server request format and support for rc/production deploys.

See https://www.colorado.edu/physics/phet/dev/html/chains/1.7.0-dev.20 as an example.

[zepumph]

The commits above should make phet-io wrappers work for the updated version id schema.

[mattpen]

The build-server requests now accomodate the new version and filename scheme.

@pixelzoom @jonathanolson @samreid - is it ok to close this issue? Should we create documentation first?

[samreid]

I don't know. What kind of documentation are you thinking of?

[jonathanolson]

Documentation (maybe in a perennial doc/*.md) about versioning is probably recommended.

[pixelzoom]

Can someone please summarize (in this issue) the format of the version id scheme that was actually implemented?

[jonathanolson]

Can someone please summarize (in this issue) the format of the version id scheme that was actually implemented?

Notes in SimVersion.js:

 * The canonical description of our general versions:
 *
 * Each version string has the form: {{MAJOR}}.{{MINOR}}.{{MAINTENANCE}}[-{{TEST_TYPE}}.{{TEST_NUMBER}}] where:
 *
 * MAJOR: Sequential integer, starts at 1, and is generally incremented when there are significant changes to a simulation.
 * MINOR: Sequential integer, starts at 0, and is generally incremented when there are smaller changes to a simulation.
 *   Resets to 0 whenever the major number is incremented.
 * MAINTENANCE: Sequential integer, starts at 0, and is incremented whenever we build with the same major/minor (but with different SHAs).
 *   Resets to 0 whenever the minor number is incremented.
 * TEST_TYPE (when present): Indicates that this is a non-production build when present. Typically will take the values:
 *   'dev' - A normal dev deployment, which goes to spot (www.colorado.edu/physics/phet/dev/html/)
 *   'rc' -  A release-candidate deployment (off of a release branch). Also goes to spot only.
 *   anything else - A one-off deployment name, which is the same name as the branch it was deployed from.
 * TEST_NUMBER (when present): Indicates the version of the test/one-off type (gets incremented for every deployment).
 *   starts at 0 in package.json, but since it is incremented on every deploy, the first version published will be 1.
 *
 * It used to be (pre-chipper-2.0) that sometimes a shortened form of the (non-'phet') brand would be added to the end
 * (e.g. '1.3.0-dev.1-phetio' or '1.3.0-dev.1-adaptedfromphet'), or as a direct prefix for the TEST_TYPE (e.g.
 * 1.1.0-phetiodev.1 or 1.1.0-phetio). We have since moved to a deployment model where there are
 * subdirectories for each brand, so this is no longer part of the version. Since this was not used for any production sim
 * builds that we need statistics from, it is excluded in SimVersion.js or its description.
 *
 * Examples:
 *
 * 1.5.0 - Production simulation version (no test type). Major = 1, minor = 5, maintenance = 0
 * 1.5.0.rc.1 - Example of a release-candidate build version that would be published before '1.5.0' for testing.
 * 1.5.0.dev.1 - Example of a dev build that would be from master.
 * 1.5.0.sonification.1 - Example of a one-off build (which would be from the branch 'sonification')

[jonathanolson]

Closing, since this versioning has been fully implemented. If anyone needs anything else, please re-open.