Improve precommit hook development experience

[samreid]

From https://github.com/phetsims/chipper/issues/1342#issuecomment-1293907234 which came from https://github.com/phetsims/chipper/issues/1325, we would like to improve the precommit hook process.

@zepumph: How should we use this tool? I often have commit hooks that take longer than 100 seconds. What is an acceptable amount of time? @zepumph - sometimes to get around commit hook timing I commit locally without hooks and then run tests before pushing. @marlitas - I thought we shouldn't be making breaking changes even locally? @jonathanolson - Yes, it is better for checking out SHAs at a certain date to not have breaks committed to master. @jonathanolson - Have the commit hooks been successfully saving us from pushing breaking changes? @pixelzoom @jbphet @jessegreenberg - Yes! @marlitas - I think it would be nice to be able to make rapid commits locally or to a branch without waiting for commit hooks. @jbphet - You could make commits to a feature branch. Then when merge to master when the changes are complete. @samreid - Lets form a sub team for this. We have log data to look through now, and need a sub team to make decisions about this. @zepumph - I waited for built tools for 18.5 minutes yesterday. The time could be longer. Sometimes run without commit hooks. @jessegreenberg - I am going to try feature branches without commit hooks when working in a single repo for a bit! @kathy-phet - After the POSE convening we can have a sub-group discuss this.

[samreid]

Yes, it is better for checking out SHAs at a certain date to not have breaks committed to master.

I used that feature this week to search for the break in https://github.com/phetsims/chipper/issues/1323#issuecomment-1295494831. Only one of the dozen samples was broken, which was great and made it easy to trace the problem.

I am going to try feature branches without commit hooks when working in a single repo for a bit!

Pros: OK to commit unstable/broken code quickly without using commit hooks. People checking out master won't get that unstable code. Cons: Potential for merge conflicts, others could be committing to master at the same time. Code won't be tested on CT (unless we add facets for that).

[samreid]

Some strategies, brainstorming and paper trail from discussions with @marlitas and @zepumph yesterday:

OK to commit broken code locally, then use a pre-push hook to make sure everything is consistent before pushing. How would that help people that basically push every commit? Would we change our workflow? What about checking out broken code by date or bisect?

Commit broken code locally, then squash (bad?) commits before push? But then you lose granularity and have to make up new commit messages.

Persist CT values and use that as a guide when git bisecting Why should we have to check with CT? Why can't everything always be green? Because it takes 30+ minutes to run all tests locally.

How can we speed up tests??? Sims launch faster. Run everything in parallel. Subset of phet-io sims.

phet-io API tests only took 25% on Michael's machine. It is not the bottleneck, especially if it runs in parallel with other things.

Would restoring project references help at all?

Other TypeScript type checking performance improvement ideas: https://github.com/microsoft/TypeScript/wiki/Performance

Should we run lints in different processes?

Can we just check in with WebStorm and ask "hey webstorm, are there any type errors or lint errors?" There is a Before Commit "analyze code" inspection?

For sims, only type check sims.

You can profile the type checker to see where the bottlenecks are.

Should we transition from precommit hooks to prepush hooks? Or can we speed up the precommit hooks? https://github.com/phetsims/chipper/issues/1269

Type check downstream https://github.com/phetsims/perennial/issues/272

Type checking takes too long to be a precommit hook: https://github.com/phetsims/chipper/issues/1160

Precommit hooks should check in with watch process https://github.com/phetsims/chipper/issues/1174

We did project references in https://github.com/phetsims/chipper/issues/1055

Table with timing is https://github.com/phetsims/chipper/issues/1241

[zepumph]

```diff Index: js/scripts/hook-pre-commit-implementation.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/scripts/hook-pre-commit-implementation.js b/js/scripts/hook-pre-commit-implementation.js new file mode 100644 --- /dev/null (date 1668034306470) +++ b/js/scripts/hook-pre-commit-implementation.js (date 1668034306470) @@ -0,0 +1,203 @@ +// Copyright 2020-2022, University of Colorado Boulder + +/** + * Runs tasks for pre-commit, including lint and qunit testing. Avoids the overhead of grunt and Gruntfile.js for speed. + * + * Should only be run when developing in master, because when dependency shas are checked out for one sim, + * they will likely be inconsistent for other repos which would cause failures for processes like type checking. + * This means when running maintenance release steps, you may need to run git commands with --no-verify. + * + * Timing data is streamed through phetTimingLog, please see that file for how to see the results live and/or afterwards. + * + * USAGE: + * cd ${repo} + * node ../chipper/js/scripts/hook-pre-commit.js + * + * OPTIONS: + * --console: outputs information to the console for debugging + * + * See also phet-info/git-template-dir/hooks/pre-commit for how this is used in precommit hooks. + * + * @author Sam Reid (PhET Interactive Simulations) + */ + +const fs = require( 'fs' ); +const puppeteer = require( 'puppeteer' ); +const withServer = require( '../../../perennial-alias/js/common/withServer' ); +const execute = require( '../../../perennial-alias/js/common/execute' ); +const getPhetLibs = require( '../grunt/getPhetLibs' ); +const getRepoList = require( '../../../perennial-alias/js/common/getRepoList' ); +const generatePhetioMacroAPI = require( '../phet-io/generatePhetioMacroAPI' ); +const CacheLayer = require( '../../../chipper/js/common/CacheLayer' ); +const phetioCompareAPISets = require( '../phet-io/phetioCompareAPISets' ); +const lint = require( '../../../chipper/js/grunt/lint' ); +const reportMedia = require( '../../../chipper/js/grunt/reportMedia' ); +const puppeteerQUnit = require( '../../../aqua/js/local/puppeteerQUnit' ); + +( async () => { + + const commandLineArguments = process.argv.slice( 2 ); + const outputToConsole = commandLineArguments.includes( '--console' ); + + const getArg = arg => { + const args = commandLineArguments.filter( commandLineArg => commandLineArg.startsWith( `--${arg}=` ) ); + if ( args.length !== 1 ) { + throw new Error( `expected only one arg: ${args}` ); + } + return args[ 0 ].split( '=' )[ 1 ]; + }; + + const command = getArg( 'command' ); + const repo = getArg( 'repo' ); + + if ( command === 'lint' ) { + // Run lint tests if they exist in the checked-out SHAs. + + // lint() automatically filters out non-lintable repos + const lintReturnValue = await lint( [ repo ] ); + outputToConsole && console.log( `Linting passed with results.length: ${lintReturnValue.results.length}` ); + process.exit( lintReturnValue.ok ? 0 : 1 ); + } + + if ( command === 'report-media' ) { + + // These sims don't have package.json or media that requires checking. + const optOutOfReportMedia = [ + 'decaf', + 'phet-android-app', + 'babel', + 'phet-info', + 'phet-ios-app', + 'qa', + 'sherpa', + 'smithers', + 'tasks', + 'weddell' + ]; + + // Make sure license.json for images/audio is up-to-date + if ( !optOutOfReportMedia.includes( repo ) ) { + + const success = await reportMedia( repo ); + process.exit( success ? 0 : 1 ); + } + else { + + // no need to check + return process.exit( 0 ); + } + } + + if ( command === 'tsc' ) { + + + // Run typescript type checker if it exists in the checked-out shas + const results = await execute( 'node', [ '../chipper/js/scripts/absolute-tsc.js', '../chipper/tsconfig/all' ], '../chipper', { + errors: 'resolve' + } ); + + results.stderr.trim().length > 0 && console.log( results.stderr ); + results.stdout.trim().length > 0 && console.log( results.stdout ); + + if ( results.code === 0 ) { + outputToConsole && console.log( 'tsc passed' ); + process.exit( 0 ); + } + else { + console.log( results ); + process.exit( 1 ); + } + } + + if ( command === 'qunit' ) { + + // Run qunit tests if puppeteerQUnit exists in the checked-out SHAs and a test HTML exists. + const qUnitOK = await ( async () => { + + const cacheKey = `puppeteerQUnit#${repo}`; + + if ( repo !== 'scenery' && repo !== 'phet-io-wrappers' ) { // scenery unit tests take too long, so skip those + const testFilePath = `${repo}/${repo}-tests.html`; + const exists = fs.existsSync( `../${testFilePath}` ); + if ( exists ) { + + if ( CacheLayer.isCacheSafe( cacheKey ) ) { + return true; + } + else { + const browser = await puppeteer.launch(); + + const result = await withServer( async port => { + return puppeteerQUnit( browser, `http://localhost:${port}/${testFilePath}?ea&brand=phet-io` ); + } ); + + await browser.close(); + + outputToConsole && console.log( `${repo}: ${JSON.stringify( result, null, 2 )}` ); + if ( !result.ok ) { + console.error( `unit tests failed in ${repo}`, result ); + return false; + } + else { + CacheLayer.onSuccess( cacheKey ); + return true; + } + } + } + + outputToConsole && console.log( 'QUnit: no problems detected' ); + return true; + } + return true; + } )(); + + process.exit( qUnitOK ? 0 : 1 ); + } + + if ( command === 'phet-io-api-compare' ) { + + //////////////////////////////////////////////////////////////////////////////// + // Compare PhET-iO APIs for this repo and anything that has it as a dependency + // + const phetioAPIOK = await ( async () => { + + const getCacheKey = repo => `phet-io-api-compare#${repo}`; + + // Test this repo and all phet-io sims that have it as a dependency. For instance, changing sun would test + // every phet-io stable sim. + const phetioAPIStable = getRepoList( 'phet-io-api-stable' ); + const reposToTest = phetioAPIStable + .filter( phetioSimRepo => getPhetLibs( phetioSimRepo ).includes( repo ) ) + + // Only worry about the ones that are not cached + .filter( repo => !CacheLayer.isCacheSafe( getCacheKey( repo ) ) ); + + if ( reposToTest.length > 0 ) { + console.log( 'PhET-iO API testing: ' + reposToTest ); + + const proposedAPIs = await generatePhetioMacroAPI( reposToTest, { + showProgressBar: reposToTest.length > 1, + showMessagesFromSim: false + } ); + + const phetioAPIComparisonSuccessful = await phetioCompareAPISets( reposToTest, proposedAPIs, {} ); + + if ( phetioAPIComparisonSuccessful ) { + reposToTest.forEach( repo => CacheLayer.onSuccess( getCacheKey( repo ) ) ); + } + + return phetioAPIComparisonSuccessful; + } + else { + console.log( 'PhET-iO API testing: no repos to test' ); + return true; + } + } )(); + + process.exit( phetioAPIOK ? 0 : 1 ); + } + + // OTHER TESTS GO HERE + + // NOTE: if adding or rearranging rules, be careful about the early exit above +} )(); \ No newline at end of file Index: js/scripts/hook-pre-commit.js IDEA additional info: Subsystem: com.intellij.openapi.diff.impl.patch.CharsetEP <+>UTF-8 =================================================================== diff --git a/js/scripts/hook-pre-commit.js b/js/scripts/hook-pre-commit.js --- a/js/scripts/hook-pre-commit.js (revision 6ce9bf076772c2c7094bfed1a1efc3a2e5fa0325) +++ b/js/scripts/hook-pre-commit.js (date 1668034388940) @@ -21,20 +21,10 @@ * @author Sam Reid (PhET Interactive Simulations) */ -const fs = require( 'fs' ); const path = require( 'path' ); -const puppeteer = require( 'puppeteer' ); -const withServer = require( '../../../perennial-alias/js/common/withServer' ); const execute = require( '../../../perennial-alias/js/common/execute' ); -const getPhetLibs = require( '../grunt/getPhetLibs' ); -const getRepoList = require( '../../../perennial-alias/js/common/getRepoList' ); -const generatePhetioMacroAPI = require( '../phet-io/generatePhetioMacroAPI' ); -const CacheLayer = require( '../../../chipper/js/common/CacheLayer' ); -const phetioCompareAPISets = require( '../phet-io/phetioCompareAPISets' ); const phetTimingLog = require( '../../../perennial-alias/js/common/phetTimingLog' ); -const lint = require( '../../../chipper/js/grunt/lint' ); -const reportMedia = require( '../../../chipper/js/grunt/reportMedia' ); -const puppeteerQUnit = require( '../../../aqua/js/local/puppeteerQUnit' ); +const _ = require( 'lodash' ); // eslint-disable-line require-statement-match ( async () => { @@ -47,168 +37,40 @@ const commandLineArguments = process.argv.slice( 2 ); const outputToConsole = commandLineArguments.includes( '--console' ); - // Run lint tests if they exist in the checked-out SHAs. - const lintOK = await phetTimingLog.startAsync( 'lint', async () => { - - // lint() automatically filters out non-lintable repos - const lintReturnValue = await lint( [ repo ] ); - outputToConsole && console.log( `Linting passed with results.length: ${lintReturnValue.results.length}` ); - return lintReturnValue.ok; - } ); - - if ( !lintOK ) { - return false; - } - - const reportMediaOK = await phetTimingLog.startAsync( 'report-media', async () => { - - // These sims don't have package.json or media that requires checking. - const optOutOfReportMedia = [ - 'decaf', - 'phet-android-app', - 'babel', - 'phet-info', - 'phet-ios-app', - 'qa', - 'sherpa', - 'smithers', - 'tasks', - 'weddell' - ]; - - // Make sure license.json for images/audio is up-to-date - if ( !optOutOfReportMedia.includes( repo ) ) { - - const success = await reportMedia( repo ); - return success; - } - else { - - // no need to check - return true; - } - } ); - - if ( !reportMediaOK ) { - return false; - } - - const tscOK = await phetTimingLog.startAsync( 'tsc', async () => { - - // Run typescript type checker if it exists in the checked-out shas - const results = await execute( 'node', [ '../chipper/js/scripts/absolute-tsc.js', '../chipper/tsconfig/all' ], '../chipper', { - errors: 'resolve' - } ); - - results.stderr.trim().length > 0 && console.log( results.stderr ); - results.stdout.trim().length > 0 && console.log( results.stdout ); - - if ( results.code === 0 ) { - outputToConsole && console.log( 'tsc passed' ); - return true; - } - else { - console.log( results ); - return false; - } - } ); - - if ( !tscOK ) { - return false; - } - - const qunitOK = await phetTimingLog.startAsync( 'qunit', async () => { -// Run qunit tests if puppeteerQUnit exists in the checked-out SHAs and a test HTML exists. - - const cacheKey = `puppeteerQUnit#${repo}`; - - if ( repo !== 'scenery' && repo !== 'phet-io-wrappers' ) { // scenery unit tests take too long, so skip those - const testFilePath = `${repo}/${repo}-tests.html`; - const exists = fs.existsSync( `../${testFilePath}` ); - if ( exists ) { - - if ( CacheLayer.isCacheSafe( cacheKey ) ) { - console.log( 'unit tests success cached' ); - return true; - } - else { - const browser = await puppeteer.launch(); - - const result = await withServer( async port => { - return puppeteerQUnit( browser, `http://localhost:${port}/${testFilePath}?ea&brand=phet-io` ); - } ); + // Ordered! + const manyPromises = [ 'lint', 'report-media', 'tsc', 'qunit', 'phet-io-api-compare' ].map( task => { + return phetTimingLog.startAsync( task, async () => { + const results = await execute( 'node', [ '../chipper/js/scripts/hook-pre-commit-implementation.js', `--command=${task}`, `--repo=${repo}`, + outputToConsole ? '--console' : '' ], '../chipper', { + errors: 'resolve' + } ); + results.stdout && console.log( results.stdout ); + results.stderr && console.log( results.stderr ); + return results.code; + } ); + } ); - await browser.close(); - outputToConsole && console.log( `${repo}: ${JSON.stringify( result, null, 2 )}` ); - if ( !result.ok ) { - console.error( `unit tests failed in ${repo}`, result ); - return false; - } - else { - CacheLayer.onSuccess( cacheKey ); - return true; - } - } - } + let resolved = false; - outputToConsole && console.log( 'QUnit: no problems detected' ); - return true; - } - return true; - } ); - - if ( !qunitOK ) { - return false; - } - - //////////////////////////////////////////////////////////////////////////////// - // Compare PhET-iO APIs for this repo and anything that has it as a dependency - // - const phetioAPIOK = await phetTimingLog.startAsync( 'phet-io-api-compare', async () => { - - const getCacheKey = repo => `phet-io-api-compare#${repo}`; - - // Test this repo and all phet-io sims that have it as a dependency. For instance, changing sun would test - // every phet-io stable sim. - const phetioAPIStable = getRepoList( 'phet-io-api-stable' ); - const reposToTest = phetioAPIStable - .filter( phetioSimRepo => getPhetLibs( phetioSimRepo ).includes( repo ) ) - - // Only worry about the ones that are not cached - .filter( repo => !CacheLayer.isCacheSafe( getCacheKey( repo ) ) ); - - if ( reposToTest.length > 0 ) { - console.log( 'PhET-iO API testing: ' + reposToTest ); - - const proposedAPIs = await generatePhetioMacroAPI( reposToTest, { - showProgressBar: reposToTest.length > 1, - showMessagesFromSim: false + // TODO!!!! Reject and use promise.all? + return new Promise( resolve => { + for ( let i = 0; i < manyPromises.length; i++ ) { + const manyPromise = manyPromises[ i ]; + manyPromise.then( exitCode => { + if ( exitCode !== 0 ) { + if ( !resolved ) { + resolved = true; + resolve(); + } + } } ); - - const phetioAPIComparisonSuccessful = await phetioCompareAPISets( reposToTest, proposedAPIs, {} ); - - if ( phetioAPIComparisonSuccessful ) { - reposToTest.forEach( repo => CacheLayer.onSuccess( getCacheKey( repo ) ) ); - } - - return phetioAPIComparisonSuccessful; - } - else { - console.log( 'PhET-iO API testing: no repos to test' ); - return true; } } ); - if ( !phetioAPIOK ) { - return false; - } - - // OTHER TESTS GO HERE - - // NOTE: if adding or rearranging rules, be careful about the early exit above // If everything passed, return true for success - return true; + // const exitCodes = await Promise.all( manyPromises ); + // return _.every( exitCodes, code => code === 0 ); } ); // generatePhetioMacroAPI is preventing exit for unknown reasons, so manually exit here

[samreid]

I switched to Promise.all, and tested with echo $?, and with early failing, and error messages. Everything seems to be working OK except for the nesting of the phet-timing-log, which now looks like this:

<!-- 11/9/2022, 4:49:08 PM -->
<hook-pre-commit repo="chipper">
  <lint>
    <report-media>
      <tsc>
        <qunit>
          <phet-io-api-compare>
          </qunit> <!-- 310ms -->
        </report-media> <!-- 335ms -->
      </tsc> <!-- 416ms -->
    </hook-pre-commit repo="chipper"> <!-- 1206ms -->

[samreid]

OK I also fixed the depth. Looks like this now:

<!-- 11/9/2022, 5:06:46 PM -->
<hook-pre-commit repo="chipper">
  <lint>
  <report-media>
  <tsc>
  <qunit>
  <phet-io-api-compare>
  </lint> <!-- 378ms -->
  </phet-io-api-compare> <!-- 375ms -->
  </report-media> <!-- 394ms -->
  </qunit> <!-- 399ms -->
  </tsc> <!-- 468ms -->
</hook-pre-commit repo="chipper"> <!-- 473ms -->

This part of the issue is ready for review.

Maybe for next steps, we could gather a few days of data for how long things are taking now, and focus on the bottleneck task?

[pixelzoom]

Slack#developer:

Chris Malley Today at 12:44 AM I noticed that there were some changes to git hooks today. When I encounter errors, I’m now seeing duplicate errors in WebStorm console, for example see below. Is it possible that something is being run twice?

0 errors in 16559ms /Users/cmalley/PhET/GitHub/natural-selection/js/common/view/NaturalSelectionTimeControlNode.ts 14:8 error 'merge' is defined but never used @typescript-eslint/no-unused-vars 17:8 error 'AssertUtils' is defined but never used @typescript-eslint/no-unused-vars 21:8 error 'Tandem' is defined but never used @typescript-eslint/no-unused-vars ✖ 3 problems (3 errors, 0 warnings) Task failed: lint, /Users/cmalley/PhET/GitHub/natural-selection/js/common/view/NaturalSelectionTimeControlNode.ts 14:8 error 'merge' is defined but never used @typescript-eslint/no-unused-vars 17:8 error 'AssertUtils' is defined but never used @typescript-eslint/no-unused-vars 21:8 error 'Tandem' is defined but never used @typescript-eslint/no-unused-vars ✖ 3 problems (3 errors, 0 warnings)

Michael Kauzmann :spiral_calendar_pad: 1 hour ago Most likely it's just duplicate logging. "lint" is in a child process now, so there are probably two spots where we report logging from the subtask. I'd just note it in https://github.com/phetsims/chipper/issues/1350

[samreid]

Not sure what else to do about this issue at the moment, @zepumph or @AgustinVallejo do you recommend further changes? Want to meet again? Check in with dev team?

[AgustinVallejo]

No comments from me! I think it's ready for dev team

[zepumph]

I think it would be awesome to meet again together in the next week or two, and perhaps work on another part of the speed up potential. Running things in parallel is working well for me so far, but I haven't made many commits in the last week, so I would like more data still.

[pixelzoom]

Can we please get the duplicate error messages fixed, reported 12 days ago in https://github.com/phetsims/chipper/issues/1350#issuecomment-1310608107? It's not only annoying, but it's actually making me miss errors.

EDIT: Move to https://github.com/phetsims/chipper/issues/1359

[zepumph]

I added a calendar meeting for 12/9 for 1 hour. Let's touch base then.

[zepumph]

From above: How can we speed up tests???

general improvement

• [x] Run each part of the pre commit hook in parallel.

phet-io-api compare test:

• [ ] Subset of phet-io sims.
• [ ] Sims launch faster.
• [ ] Smarter about what repos need to run this test.

tsc

• [ ] Attempt to use project references again

[zepumph]

We worked a bit more on project references https://github.com/phetsims/chipper/issues/1356 and are still figuring out how to handle mixins.

[samreid]

@marlitas said #1325 relates to this. I added assignees for this subgroup. @zepumph said it would be good to schedule regular meetings on this topic until it is better under control.

[jessegreenberg]

I just had a particularly slow experience while using pre-commit hooks with PhET-iO API validation:

1) Run pre-commit hooks on change in scenery-phet (~3 minutes )
2) Hooks identified a PhET-iO API break, so I regenerated PhET-iO API (~1.5 minutes)
3) Re-run pre-commit hooks (2 minutes)
4) Commit changes (0.5 minutes)
5) Ready to push! ...push failure -> merge required -> pull scenery-phet with --rebase (0 minutes)
6) Run pre-commit hooks...failure because the scenery-phet change is incompatible with my axon (~2 minutes)
7) Stash my changes, pull all, re-apply my changes, restart transpiler (0.5 min)
8) Re-generate PhET-iO API (1.5 minutes)
9) Run pre-commit hooks (~2 minutes)
10) commit (0.5 minutes)
11) Successful push

Id be interested in looking joining this sub-group issue or looking into workflow alternatives.

[zepumph]

This will need time carved out to work on it. I have recommended it for iteration. Unassigning.

[jessegreenberg]

For sim code I have been working in feature branches mostly for the past few weeks and it has improved my productivity. In https://github.com/phetsims/perennial/issues/276 we made it so pre-commit hooks only run for master. Removing my assignment until we create dedicated time for this.

[samreid]

You can use node chipper/js/scripts/precommit-hook-multi.js to check all precommit hooks on the repos with changes.

[jonathanolson]

In WebStorm/Intellij, I have the following external tool defined that lets me in-parallel lint/tsc. It runs pretty quickly now, and will link any failed lines.

Setup here:

[zepumph]

1. CTQ needs to do all the work the git hooks do.

[pixelzoom]

The above commits by @zepumph have a markdown formatting problem, reported in https://github.com/phetsims/aqua/issues/210.

[zepumph]

Today we had another dev meeting discussion on phet-io api checking in pre commit hooks. Here is a copy of the discussion:

• MK: Let’s discuss git hook + PhET-iO API checking.
  • MK: maybe we should prune the list done by the pre commits, but keep CTQ running everything, so we know eagerly what is wrong.
  • MK: Many new sims have been added to the stable list. Committing to common code requires a lot of processing time for precommit hooks to run.
  • MK: Can we trim the list of processes that we really care about to run?
  • MK: Can we keep a shorter list of sims that have been recently updated to reduce the precommit burden?
  • JO: API changes coming in can interfere with the scripts I am running
  • CM: If everything is not being checked, we need to agree to not push to common code at the end of the day unless we are checking CT.
    • MK: I think using CTQ would be a way that we can catch these
    • MS: but we can break it not just on common code right?
      • MK: The goal is for commit to sims to not prevent other sims from committing
  • MS: I have been running into breaking API changes from other commits not in the sim I was working on
    • MS will reach out
  • MK: We do not want to get rid of API checking as part of precommit hooks
  • MK: What if precommit hooks had 5 sims? Would that affect peoples’ work?
    • JG: I would feel most comfortable if they are all tested, but if checked on CTQ I wouldn’t feel the need to do so locally
    • ~~MS: I would not want to remove sims from precommit hooks if I’m working in common code ~~I was confused on what we were talking about.
  • CM: We could make it a norm to regenerate APIs if working in a PhET-IO sim, see how much gets caught by CT.
  • MK: Could we just run precommit hooks in the repo we are working on?
    • CM: I generally regenerate the API in the repo I’m working in many times before committing, and so I haven’t gotten much benefit when working in a sim
  • Proposals:
    • 1. Remove PhET-iO API checking from precommit hooks, but keep on CTQ
         • MK, JG, CM, JO
    • 2. Do nothing, no changes
         • MS, AV, MB
    • 3. Pre commit hooks only look at a subset of stable phet-io sims, yielding less checks (less redundancy?) and more risk before commit
         • (none)
  • JG: Note that using a branch helps defer the API checks to the branch merge, instead of each commit.
    • MK: I would love to have a set of repos devoted to general common code changes that is tested by CT.
  • MS: Can we opt into having phet-io api checks in pre-commit hooks if we want?
    • MK: Would require repo-specific install within WebStorm
    • MS: Could I pass an option through in the console?
      • CM: There is no way that I know of in WebStorm, other than going out to a console
        • MS: I have an alias for precommit hooks to run in the console. Could I have an option for API checking in that command?
          • MK, CM: Yes, this could work
  • MK: No immediate next steps recommended. Summer would not be a good time to change this with vacations.

---

Basically, it seems like we may want to allow git hooks to be more flexible, and rely on running more checks manually before commit, or face the wrath of CTQ.