phetsims / aqua

Automatic QUality Assurance
MIT License
2 stars 4 forks source link

CTQ formatting problem for "error testing phetioCompare" #210

Open pixelzoom opened 6 months ago

pixelzoom commented 6 months ago

For https://github.com/phetsims/chipper/issues/1350, @zepumph made a number of commits to "compare all stable sims for CTQ". Those commits are linked just below https://github.com/phetsims/chipper/issues/1350#issuecomment-2086352580.

It looks like there's a markdown problem with how these CTQ notifications appears in Slack. For example:

screenshot_3289



Other CTQ notifications are formatted like this:

screenshot_3290
zepumph commented 6 months ago

The formatting looks right, but it seems to have come in two different messages for some reason:

First message:

CTQ failing: -three backtics- error testing phetioCompare: buoyancy-basics BREAKING PROBLEMS buoyancyBasics.compareScreen.model.blockSets.sameDensity.blockA.materialProperty._data.initialState differs. Expected: {"units":null,"validValues":null,"value":{"custom":true,"customColor":null,"density":400,"hidden":false,"identifier":null,"liquidColor":null,"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"staticCustomColor":{"a":1,"b":71,"g":75,"r":235},"staticLiquidColor":null,"tandemName":"custom","viscosity":0.001}} actual: {"value":{"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"identifier":null,"tandemName":"custom","density":400,"viscosity":0.001,"custom":true,"hidden":false,"staticCustomColor":{"r":235,"g":75,"b":71,"a":1},"customColor":null,"staticLiquidColor":{"r":235,"g":75,"b":71,"a":1},"liquidColor":null},"validValues":null,"units":null} buoyancyBasics.compareScreen.model.blockSets.sameDensity.blockB.materialProperty._data.initialState differs. Expected: {"units":null,"validValues":null,"value":{"custom":true,"customColor":null,"density":400,"hidden":false,"identifier":null,"liquidColor":null,"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"staticCustomColor":{"a":1,"b":175,"g":105,"r":67},"staticLiquidColor":null,"tandemName":"custom","viscosity":0.001}} actual: {"value":{"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"identifier":null,"tandemName":"custom","density":400,"viscosity":0.001,"custom":true,"hidden":false,"staticCustomColor":{"r":67,"g":105,"b":175,"a":1},"customColor":null,"staticLiquidColor":{"r":67,"g":105,"b":175,"a":1},"liquidColor":null},"validValues":null,"units":null} buoyancyBasics.compareScreen.model.blockSets.sameMass.blockA.materialProperty._data.initialState differs. Expected: {"units":null,"validValues":null,"value":{"custom":true,"customColor":null,"density":500,"hidden":false,"identifier":null,"liquidColor":null,"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"staticCustomColor":{"a":1,"b":47,"g":52,"r":219},"staticLiquidColor":null,"tandemName":"custom","viscosity":0.001}} actual: {"value":{"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"identifier":null,"tandemName":"custom","density":500,"viscosity":0.001,"custom":true,"hidden":false,"staticCustomColor":{"r":219,"g":52,"b":47,"a":1},"customColor":null,"staticLiquidColor":{"r":219,"g":52,"b":47,"a":1},"liquidColor":null},"validValues":null,"units":null} buoyancyBasics.compareScreen.model.blockSets.sameMass.blockB.materialProperty._data.initialState differs. Expected: {"units":null,"validValues":null,"value":{"custom":true,"customColor":null,"density":2500,"hidden":false,"identifier":null,"liquidColor":null,"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"staticCustomColor":{"a":1,"b":73,"g":39,"r":21},"staticLiquidColor":null,"tandemName":"custom","viscosity":0.001}} actual: {"value":{"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"identifier":null,"tandemName":"custom","density":2500,"viscosity":0.001,"custom":true,"hidden":false,"staticCustomColor":{"r":21,"g":39,"b":73,"a":1},"customColor":null,"staticLiquidColor":{"r":21,"g":39,"b":73,"a":1},"liquidColor":null},"validValues":null,"units":null} buoyancyBasics.compareScreen.model.blockSets.sameVolume.blockA.materialProperty._data.initialState differs. Expected:

Then another message

{"units":null,"validValues":null,"value":{"custom":true,"customColor":null,"density":400,"hidden":false,"identifier":null,"liquidColor":null,"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"staticCustomColor":{"a":1,"b":71,"g":75,"r":235},"staticLiquidColor":null,"tandemName":"custom","viscosity":0.001}} actual: {"value":{"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"identifier":null,"tandemName":"custom","density":400,"viscosity":0.001,"custom":true,"hidden":false,"staticCustomColor":{"r":235,"g":75,"b":71,"a":1},"customColor":null,"staticLiquidColor":{"r":235,"g":75,"b":71,"a":1},"liquidColor":null},"validValues":null,"units":null} buoyancyBasics.compareScreen.model.blockSets.sameVolume.blockB.materialProperty._data.initialState differs. Expected: {"units":null,"validValues":null,"value":{"custom":true,"customColor":null,"density":2000,"hidden":false,"identifier":null,"liquidColor":null,"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"staticCustomColor":{"a":1,"b":82,"g":44,"r":23},"staticLiquidColor":null,"tandemName":"custom","viscosity":0.001}} actual: {"value":{"name":{"phetioID":"buoyancyBasics.general.model.strings.densityBuoyancyCommon.material.customStringProperty"},"identifier":null,"tandemName":"custom","density":2000,"viscosity":0.001,"custom":true,"hidden":false,"staticCustomColor":{"r":23,"g":44,"b":82,"a":1},"customColor":null,"staticLiquidColor":{"r":23,"g":44,"b":82,"a":1},"liquidColor":null},"validValues":null,"units":null} -three backtics-

pixelzoom commented 5 months ago

This is still a problem as of 6/17/2024:

screenshot_3366
zepumph commented 1 month ago

I just saw an error on the server that says:

|ct-quick | 2024-09-26T11:48:56: [WARN]  bolt-app message_truncated 0 [ 'message_truncated' ]
1|ct-quick | 2024-09-26T11:48:56: [WARN]  bolt-app Your message was truncated but still posted. The `text` field accepts up to 40,000 characters.