CT Assertion failed - Githubissues

KatieWoe commented 5 years ago

energy-skate-park-basics : fuzz : require.js : run
Query: brand=phet&ea&fuzz&memoryLimit=1000
Uncaught Error: Assertion failed
Error: Assertion failed
    at window.assertions.assertFunction (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/assert/js/assert.js:22:13)
    at SkaterState.updateThermalEnergy (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/energy-skate-park/js/energy-skate-park/common/model/SkaterState.js?bust=1557312673873:236:17)
    at PlaygroundModel.stepGround (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/energy-skate-park/js/energy-skate-park/common/model/EnergySkateParkModel.js?bust=1557312673873:464:22)
    at PlaygroundModel.stepModel (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/energy-skate-park/js/energy-skate-park/common/model/EnergySkateParkModel.js?bust=1557312673873:1298:70)
    at PlaygroundModel.step (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/energy-skate-park/js/energy-skate-park/common/model/EnergySkateParkModel.js?bust=1557312673873:398:31)
    at https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/joist/js/Sim.js?bust=1557312673873:206:22
    at Action.execute (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/axon/js/Action.js?bust=1557312673873:177:20)
    at EnergySkateParkBasicsSim.stepSimulation (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/joist/js/Sim.js?bust=1557312673873:914:33)
    at EnergySkateParkBasicsSim.stepOneFrame (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/joist/js/Sim.js?bust=1557312673873:895:14)
    at EnergySkateParkBasicsSim.runAnimationLoop (https://bayes.colorado.edu/continuous-testing/snapshot-1557296817928/joist/js/Sim.js?bust=1557312673873:878:14)
id: Bayes Chrome
Approximately 5/8/2019, 12:26:57 AM
energy-skate-park-basics : phet-io-tests : assert
8 out of 8 tests passed. 0 failed.

Approximately 5/8/2019, 12:26:57 AM
energy-skate-park-basics : phet-io-tests : no-assert
8 out of 8 tests passed. 0 failed.

Approximately 5/8/2019, 12:26:57 AM

samreid commented 5 years ago

There have been a number of issues like this. Extremely rare, and difficult or impossible to create locally. Energy Skate Park: Basics was developed before we had fuzz testing or CT. In prior issues, @jessegreenberg and I worked together to add information to the assertion messages to help us work backwards to see what is happening, but these are difficult issues to investigate. Some ideas for proceeding:

Schedule time for @samreid and/or @jessegreenberg to investigate this issue and others of its kind.
Shut off assertions like this when ?fuzz is enabled. For a particular class of problems in PhET-iO, that was the appropriate solution.
Request time from QA team to try to find steps to reproduce this problem manually, then hand it over to @samreid and @jessegreenberg.
Ignore it altogether.

@ariel-phet what do you recommend for this issue and others like it?

ariel-phet commented 5 years ago

@samreid I would like to discuss a bit at dev meeting to get a better perspective. I am leaning towards your second suggestion

Shut off assertions like this when ?fuzz is enabled. For a particular class of problems in PhET-iO, that was the appropriate solution.

But want to make sure there is consensus.

jessegreenberg commented 5 years ago

Here is the original issue with notes about work done for this so far: https://github.com/phetsims/energy-skate-park/issues/45

pixelzoom commented 5 years ago

I'm not in favor of turning off assertions when ?fuzz is enabled, or any other "solution" that hides errors. It's better to live with errors than to hide them, and address them when we have time. If we can't find time, then there is a bigger problem than the errors.

samreid commented 5 years ago

After discussion @ariel-phet said, after GQIO and diffraction, can investigate ESP CT fails with @jessegreenberg for 1/2 day to a day.

jessegreenberg commented 5 years ago

@jonathanolson recommended recording the sim so that we have the reproducible events that lead to this issue.

samreid commented 5 years ago

Unassigning, we are pursuing solutions in https://github.com/phetsims/energy-skate-park/issues/45

samreid commented 5 years ago

@jessegreenberg can this issue be closed? Are all of the concrete problems noted in other issues?

jessegreenberg commented 4 years ago

Yes this can be closed. This assertion in particular has a proposed fix and is being tracked in #45.

phetsims / energy-skate-park-basics

CT Assertion failed #458