Build PhET and PhET-iO brands together

[samreid]

In https://github.com/phetsims/chipper/issues/560 we decided we want to investigate building both brands simultaneously.

@jonathanolson and @zepumph and @samreid will work together to investigate how to make this happen.

Questions:

• how to make the build process not run too slowly? (RJS build takes about 15 seconds for me on FAMB). May be possible to reuse build artifacts, but that may be too complex.
• Will we still have 2 dependcies.jsons? Yes, one per subdirectory.
• what will the directory structure be like? @pixelzoom likes the idea of one subdirectory per brand.
• can we use build directory structure for publishing? This may impact analytics and publishing steps.
• should we turn phet-io into a flag (can be combined with brands) instead of an either-or brand? Or specifying brand='phet-io' is one way to turn on this flag?
• making sure the build still works properly for 3rd party developers that don't have the phet-io directory checked out.
• This will impact testing. Instead of putting "/build" in the path, you would have to put "/build/phet" in the path.

We'll try to look into this before May 4.

[samreid]

I have a faint recollection that someone within the last 2 weeks was trying to run 2 requirejs builds and we needed to clear it out somehow? Maybe it was @zepumph? I'm asking because it could impact how we can use requirejs during chipper steps.

[pixelzoom]

It was not me.

[samreid]

@kathy-phet can we put the "PhET" and "PhET-iO" builds side by side like this?

https://phet.colorado.edu/sims/html/states-of-matter/1.0.2/phet https://phet.colorado.edu/sims/html/states-of-matter/1.0.2/phet-io

[kathy-phet]

Not with the current licensing and contract statements - these say that the sims at the phet.colorado.edu website are CC-BY. It was the easiest way to be clear about which sims were available under our CC-BY license.

So if you want to do this, we will need to revisit. Why do you want to do this approach -- since its not at phet-io? It seems like it would be confusing to clients, and could cause some regular users to run the phet-io sims out of compliance with the license.

[kathy-phet]

Why not https://phet.colorado.edu/sims/html/states-of-matter/1.0.2/ https://phet-io.colorado.edu/sims/html/states-of-matter/1.0.2/

[pixelzoom]

Builds on spot could presumably live side-by-side in the same directory, e.g.:

dev/html/states-of-matter/1.0.2/
  states-of-matter_en.html
  states-of-matter_en-phetio.html
  ...

But "grunt deploy-production" should deploy them to different production servers, as @kathy-phet indicated in https://github.com/phetsims/chipper/issues/567#issuecomment-296288133.

[pixelzoom]

Or if we put brands in subdirectories on spot, as discussed in yesterday's dev meet:

dev/html/states-of-matter/1.0.2/
  phet/
    states-of-matter_en.html
    ...
  phet-io/
    states-of-matter_en-phetio.html
    wrappers/
    ...

[samreid]

So if you want to do this, we will need to revisit. Why do you want to do this approach

We are looking at ways to unify the phet and phet-io development/build/test/deployment process. One recommendation from yesterdays development meeting was that the build process should build both brands during each build.

I'm mainly trying to enumerate constraints and see what's possible.

[samreid]

Brainstorming: Perhaps we can publish phet-io and phet simulations on spot and phet-io.colorado.edu. If the only restriction is not having the io simulations accessible from phet.colorado.edu, perhaps they could be blacklisted with an apache directive. Then the same build/deploy process can be used for all 3 sites.

[samreid]

Why not https://phet.colorado.edu/sims/html/states-of-matter/1.0.2/ https://phet-io.colorado.edu/sims/html/states-of-matter/1.0.2/

That is (aside from the html/) Plan A, which involves some additional complexity to the build and deploy process. We may eventually go with it, but first it would be good to understand alternatives.

[zepumph]

@jonathanolson, @samreid and I met for a bit this morning. We didn't decide on anything concrete, but rather just pursued different ideas and brainstormed pros and cons.

When thinking through the task of combining the phet-io and phet build processes together, a spectrum we thought about was how much to manipulate the build process (meaning the build server too).

On one side, a solution would be to just update the instructions to manually build the phet and phet-io sims one after another. This would be a simple(ish) implementation because the basic problem would be just combining release branches (1.1 with 1.1-phetio) so that when you manually build both it is always from the same shas. This is a bummer though because it is a hassle to add so many steps to the build process, especially if you don't care about one of the brands you are building.

We could also fork grunt into two sub processes (not a popular choice because there are potentially unforeseen problems in propagating errors/feedback through), one for the phet build and a fully separate one for the phet-io build.

The other side would be dig deep into the grunt build and make the default grunt command build a phet and a phet-io version, and put them into differently branded sub directories in the build/ dir.

Either way, maintaining and updating the dependencies.json file is a large part of the problem. If we have two fully separate dependency.json files, then it would not have to manipulate the grunt command as much (see deployUtil.commitAndPushDependeciesJSON for details), but that could be a slippery slope, having one dependencies.json file per brand. The other option here would be to just have one dependencies file. This would work because phet-io dependencies is a superset of phet dependencies, so we could just always checkout shas for repos like phet-io and phet-io-website, even when building normal phet brand sims. (I think we seemed to prefer this strategy)

A few other talking points:

• How do we want to handle this process in the context of 3rd party developers. If we make phet builds depend on the phet-io repo, then we need to figure out the best way around problems if someone doesn't have access to these private repos.
• @samreid brought up the point of balancing the cost to manipulate the build process, versus the cost to adding annoyance/time to the build process. One argument is that if there are a few hundred builds in the next x amount of time, the "adding instructions for developers to manually do" method could add a few minutes to each of these builds in an annoying way, but it may be better than hours upon hours of tearing the build process apart. On the other side. If it turns out that this more manual step ends up needing changes and chipper and perennial anyways (which I think is inevitable), then perhaps we should try to do it right, in the most maintainable and forward thinking (least hackish) manner.

Please add your thoughts as you see fit, to this issue or below. @samreid and @jonathanolson and I are going to sleep on it, talk about this again soon, try to solidify a few concrete solutions with pros and cons, and then bring it up to developers on Thursday.

[zepumph]

This wasn't ready for discussion in today's dev meeting. We should shoot to have a conversation about this before next dev meeting.

[samreid]

Here are some upcoming availabilities:

Tuesday May 2, 1pm-3pm Mountain Time Wednesday May 3, 9am-3pm Mountain Time

[zepumph]

Both of those work well for me. @jonathanolson what's good for you?

[jonathanolson]

Tuesday sounds great to me.

[zepumph]

Alright sounds good. How about 1pm tomorrow then?

[samreid]

1pm-1:50pm Tuesday works OK for me. I have another meeting at 2pm I'll need to prepare for.

[samreid]

We chatted about this yesterday and @pixelzoom joined in. It seems important to only have one maintenance branch per version which includes all the brands. That is, instead of having 1.1-phetio and 1.1, we would have only 1.1 and it would support all brands.

To decide:

• [ ] should there be only one dependencies.json which identifies dependencies for all brands (as a union?) . This may be best because it enumerates everything the repo depends on to build all the brands. The main drawback seems to be that it would include private repo dependencies which not everyone can check out, so we may need to update our tools around that.
• [ ] should the build process always output phet and phet-io brand, or should the instructions say to run grunt twice?
• [ ] If we want to automate it, to what degree should we rewrite everything vs leverage our existing tools? For instance, if we always want to build both brands, but we don't want to invest in a chipper/perennial redesign, we could write a bash script that leverages the existing grunt tasks.

[pixelzoom]

Off the top of my head....

should there be only one dependencies.json which identifies dependencies for all brands (as a union?) .

Having a dependencies file per brand feels like the way to go. Add the brand name as a file suffix, e.g. dependencies-phet-io.json (or dependencies-phetio.json, depending on what the brand name really is). Absence of a brand suffix could indicate phet brand, but it might be clearer (and simplify things) to have dependencies-phet.json.

Speaking of brand suffixes... This might also be the time to consider whether the HTML file for PhET brand should be (e.g.) beers-law-lab_en-phet.html instead of beers-law-lab_en.html. It would certainly simplify things to treat all brands uniformly.

should the build process always output phet and phet-io brand, or should the instructions say to run grunt twice?

This is the wrong question, it implies that there will only be phet and phet-io brands. If we're looking at longterm solutions, we need to stop thinking of this as "phet and phet-io". That said...

The build task already has a --brand option. Generalize that to --brands, similar to --locales. Absence of --brands is equivalent to --brands=*. We'll need a way to detect or enumerate the supported brands for each sim.

If we want to automate it, to what degree should we rewrite everything vs leverage our existing tools?

If PhET is really committed to brands in general, and PhET-iO in particular, then I don't think there's any argument that PhET needs a well-defined process and automated build tools to support those things. So in the longterm, there is no way to avoid incurring the large cost of modifying/developing tools and processes to support those things -- it's a question of when. "Making do" with existing tools may allow PhET to avoid the larger cost for awhile. But it will almost certainly increase the cost/complexity of a longterm solution (more manual work for developers in the meantime, more legacy "intermediate solutions" to support in the future, ...)

[samreid]

I agree with the majority of your comment above, the only part I am curious about is why to have different dependencies.json. It seems like that would provide the opportunity for SHAs to get out of sync. Having one dependencies.json would guarantee that SHAs are all in sync and there is no opportunity for error (same reason as we wanted to eliminate multiple maintenance branches).

I see two main options here:

• One dependencies.json with the union of all brand shas, and if you try to check out shas for private repos, then it ignores them silently
• One dependencies.json which tracks the shas, then other metadata (either in the build.json or dependencies-brand.json files) to track which repos each brand uses.

[pixelzoom]

There are situations where the shas should be out of sync for each brand. Suppose you make a PhET-iO fix and want to publish only the phet-io brand in maintenance release 1.0.4. Run grunt --brand=phet-io and deploy the artifacts. dependences-phetio.json contains the dependencies, and they are at this point different than dependences-phet.json (because you didn't publish phet brand). Sometime later you have a general fix that requires publishing but phet and phet-io brands. The maintenance release will have version id 1.0.5. Run grunt --brand=* and deploy the artifacts. Now dependences-phet.json and dependences-phetio.json are identical.

[pixelzoom]

This is probably also a good place to recommend that PhET consider having someone whose primary responsibility is supporting the building and deployment of PhET products. PhET's requirements in this area are more complicated (and unique) than the majority of organizations that I've consulted for in the past. And 100% of those organizations had someone whose full-time responsibility was to support building and deploying products. The introduction of brands and PhET-iO has increased the complexity of this significantly, and it's unreasonable to expect that this can be handled by developers whose primary responsibility is sim development.

[pixelzoom]

To summarize the above comment: Supporting build and deploy is no longer a part-time job.

[samreid]

If we have one maintenance branch for multiple brands, then why would we want that branch to have conflicting dependencies? For instance, if Faraday's Law 1.1 branch had dependencies-phet.json with joist sha ac123e why would we want to permit dependencies-phetio.json to have joist sha fea238? That would mean that when switching brands we would have to check out different shas, which would significantly complicate the process. The process would be simpler if we constrain the shas to match, and always build all supported brands.

Suppose you make a PhET-iO fix and want to publish only the phet-io brand in maintenance release 1.0.4

In that case, the PhET features must be tested, so why not republish the PhET brand sim as well?

[pixelzoom]

In that case, the PhET features must be tested, so why not republish the PhET brand sim as well?

The update is irrelevant to the phet brand, but the end-user will be notified that an update is available. There has been push-back to that in the past.

[samreid]

What if we still build and test both versions, but we only publish the PhET-iO brand one? That will keep the shas in sync without sending out updates.

[pixelzoom]

I should clarify something else here. I started https://github.com/phetsims/chipper/issues/567#issuecomment-298967526 with:

Off the top of my head....

... because I don't feel like I have the information (or time, or interest) needed to make good decisions about build process and tools. I don't know what PhET's longterm plans are, I don't know how PhET-iO products are delivered, I don't know the desired relationship between PhET and PhET-iO branded products, ....

So take that into consideration with any of my recommendations related to this topic.

[samreid]

I don't know what PhET's longterm plans are, I don't know how PhET-iO products are delivered, I don't know the desired relationship between PhET and PhET-iO branded products

I'm sure you aren't the only one! It would be great to discuss with @kathy-phet and @ariel-phet so we can get a shared vision.

[jonathanolson]

What if we still build and test both versions, but we only publish the PhET-iO brand one? That will keep the shas in sync without sending out updates.

Applying the fixes (patches to dependencies.jsons) to both phet and phet-io brands (and then conditionally not deploying a specific brand if no change is needed) sounds better than having them diverge.

[pixelzoom]

Another question is whether PhET build tools need to be usable outside of PhET. If yes, then having all dependencies in a single dependencies.json will require extra logic to ignore repos that are not actually needed to build the artifact. For example, brand=phet-io requires some repos that brand=phet doesn't require.

[zepumph]

This issue is tagged for dev meeting, if you want a decent summary of the issue see https://github.com/phetsims/chipper/issues/567#issuecomment-298941460

[samreid]

Questions that would be good to discuss with @kathy-phet and @ariel-phet which will help use decide how to approach our build tools:

1. When should we start "instrumenting sims as we go"?
2. When will we start publishing the PhET and PhET-iO brands at the same time?
3. When do we hope to have all of our HTML5 sims PhET-iO instrumented?

[samreid]

@kathy-phet says:

When should we start "instrumenting sims as we go"?

we wanted the approach to stabilize before scaling up on this. May not be quite there, especially with respect to games like Build an Atom. Still need to identify the gaps on what sim features need more attention.

@jbphet says dynamic creation and deletion of objects is a bit tricky for instrumentation and may need to be changed.

According to @pixelzoom @ariel-phet said Equality Explorer is a rush, so no time to instrument as we go for that one.

@kathy-phet says don't instrument as we go now, but we can work on instrumenting already developed sims. Aim at getting the easy ones done so we have a larger collection.

@pixelzoom asks: should we invest now in upgrading our tools? Or cobble something together?

@kathy-phet says: it would be helpful to have a good time estimate

@pixelzoom agrees, it is unclear how long it would take to redo chipper.

[samreid]

@kathy-phet asks what are the costs of the current procedure

@pixelzoom: there are manual steps to keep the shas in sync, and keep the branches in sync. The parts are test phase, maintenance phase, patching and dealing with existing branches. What if we publish a PhET update when there are only internal (or PhET-iO related changes?). Will we have other brands that we need to support in the future?

@jonathanolson: right now there are lots of manual steps to manage everything. In the future, we may have another process that can help with this automatically.

@pixelzoom: perennial needs to know when we make build API changes. Perennial would need to support both.

[samreid]

When do we hope to have all of our HTML5 sims PhET-iO instrumented?

@kathy-phet says sooner is better but not practical. Depends on fund raising. Would be good to have a few more instrumented by Fall. Fall is a good target for a few more, then 50% sims by Fall 2018. If clients need certain sims instrumented, we can focus on them.

[samreid]

@pixelzoom: we are still in beta for PhET-iO. By the time we are out of beta, we should have well-defined tools and processes.

@kathy-phet: I'm not worried about bothering PhET users with PhET-iO changes, it doesn't seem too intrusive.

@pixelzoom: we could build both brands but not deploy both.

@kathy-phet: if PhET-iO and PhET are on same branches, then the PhET brand should still work.

[samreid]

When will we start publishing the PhET and PhET-iO brands at the same time?

@kathy-phet says: I'm fine with that.

[samreid]

@jonathanolson: we'll have one branch for both, and always build both at build time

[samreid]

Do we need to support 3rd parties being able to check out and run PhET sims?

@kathy-phet: I recommend against 3rd parties forking PhET code because it makes it difficult for us to provide maintenance releases to them. I am more often more inclined to put features into the main sim so that we can maintain it more simply.

@kathy-phet: At this point, I'm fine with 3rd parties not being able to build things because of the missing private repos.

@jonathanolson: Perennial could assume we have PhET-iO, but grunt shouldn't.

@kathy-phet: sounds fine to me

[samreid]

Would be great to discuss with @ariel-phet soon. @ariel-phet please read through the comments and recommendation and let's chat at an upcoming developer meeting.

[kathy-phet]

Perhaps goal of having Build an Atom game instrumented by end of summer -- so we can figure out whether PhET-iO is working for games by then?

[zepumph]

Adding to dev meeting for next week, when we can talk with @ariel-phet.

[samreid]

@ariel-phet suggests that after CCK is out, we can schedule a block of time with @samreid, @zepumph and @pixelzoom and @jonathanolson to iron this out.

@pixelzoom also pointed out that @kathy-phet wanted to confirm that current strategies work OK for a game before we ramp up.

@ariel-phet will also read through this issue and comment later on.

[zepumph]

Parent issue for chipper 2.0: https://github.com/phetsims/chipper/issues/596. Unassigning until we have time to work on that.

[zepumph]

Marking for dev meeting to begin discussing process. Then we will bring big picture questions to @kathy-phet and then carry on with chipper:2.0 next month.

[ariel-phet]

Assigning to @zepumph to distill this issue and perhaps open a new distilled issue, or break out the key points into separate issues and close.

[zepumph]

Here is my summary of this issue. In general I will create new issues for different parts of this. Bold parts are headers.

First we talked about what the build artifacts will look like in the build/ dir. This assumes that we are building both(all) every time, but I don't think that was settled at all. A. Build each brand into a single directory, each with their own dependencies file:

dev/html/states-of-matter/1.0.2/
  phet/
    states-of-matter_en.html
    ...
  phet-io/
    states-of-matter_en-phetio.html
    wrappers/
    ...

B. same build folder like for spot/testing purposes:

dev/html/states-of-matter/1.0.2/
  states-of-matter_en.html
  states-of-matter_en-phetio.html

Then we talked about amount of work to be done to the build process Spectrum: how much do we manipulate the build process to achieve this goal?

• Side 1 - easy to implement, not very robust: Just combine phet-io/phet release branches, and still manually build/deploy the one you need based on the sim_deployment guidelines.

• -- work on the automated build/deploy process to limit the steps in the sim_deployment guidelines, but you still build/deploy each brand separately

• -- fork the grunt process into two separate sub-processes, one for each brand

• Side 2 - hard to implement, perhaps fast/simple to run: Dig deep into the grunt build and make the default grunt command build a phet and a phet-io version. Maybe even using the same build artifacts so that it isn't just grunt; grunt --brand=phet-io, but much more efficient.

DECIDED: one maintenance branch per version

How do handle the dependencies files:

• One for each brand:
  • then it would not have to manipulate the grunt command as much (see deployUtil.commitAndPushDependeciesJSON for details), but that could be a slippery slope, having one dependencies.json file per brand.
  • It seems like that would provide the opportunity for SHAs to get out of sync. (same reason as we wanted to eliminate multiple maintenance branches).
• Only one single dependencies file:
  • This would work because phet-io dependencies is a superset of phet dependencies, so we could just always checkout shas for repos like phet-io and phet-io-website, even when building normal phet brand sims. (I think we seemed to prefer this strategy)
    • Two options of how to have 1 dependencies file:
    • One dependencies.json with the union of all brand shas, and if you try to check out shas for private repos, then it ignores them silently
    • One dependencies.json which tracks the shas, then other metadata (either in the build.json or dependencies-brand.json files) to track which repos each brand uses.

Build and Deploy each time?

• Applying fixes (patches to dependencies.jsons) to both phet and phet-io brands (and then conditionally not deploying a specific brand if no change is needed) sounds better than having them diverge. (many thumbs ups for that comment). This means that we build and test all brands, but only deploy what we need.

Other questions not really answered:

• What about third party developers? https://github.com/phetsims/chipper/issues/567#issuecomment-298995619
• Should the build process always output phet and phet-io brand, or should the instructions say to run grunt twice?
  • @pixelzoom thought: "This is the wrong question, it implies that there will only be phet and phet-io brands. If we're looking at longterm solutions, we need to stop thinking of this as "phet and phet-io". That said... The build task already has a --brand option. Generalize that to --brands, similar to --locales. Absence of --brands is equivalent to --brands=*. We'll need a way to detect or enumerate the supported brands for each sim."
• Should PhET have someone whose primary responsibility is the build tools? https://github.com/phetsims/chipper/issues/567#issuecomment-298974310
• Will we have other brands that we need to support in the future?

[zepumph]

Now I am going to summarize the summary:

I'm pretty sure these are decided, but it will still be good to discuss and make sure:

• 1 dependencies.json file, that holds a superset of all shas needed for all brands
• 1 maintenance release branch per version, that holds all brands with that version.
• Build/test every brand in a release branch, but only deploy the brand you need to.
• Should the default grunt task build all brands every-time by default: yes, or at least it should be able to build all brands with options like brands=*

Not really decided at all:

• What about third party developers, https://github.com/phetsims/chipper/issues/567#issuecomment-298995619? Do we update the build process to be graceful and fallback to just phet brand if there aren't phet-io specific repos checked out. Is this the case for tools like grunt checkout-shas also? Especially since we are probably going with 1 dependencies.json file for all brands.
• Should PhET have someone whose primary job is the build tools? #625
• Will we have other brands that we need to support in the future?

Assigning back to @ariel-phet to review and tagging for dev meeting.

[pixelzoom]

Self assigning so that I remember to review comments on this issue that occurred since 11/17/17.

[pixelzoom]

Re @zepumph summary in https://github.com/phetsims/chipper/issues/567#issuecomment-346512461 ...

(1) The summary seems to be related to an unidentified discussion ("we talked about") that took place. It would be nice to have context.

(2) The directory structures don't match the decisions made in 11/16/17 dev meeting, which were approved by @kathy-phet. See "desired directory structure" in https://github.com/phetsims/chipper/issues/560#issuecomment-342227691. Why are we changing the directory structures?

(3) I see:

DECIDED: one maintenance branch per version

If this means "one maintenance branch per {{MAJOR}}.{{MINOR}} pair" then agreed. If not, then please clarify, because this is not what was decided previously.