CTQ can't handle one really bad codebase change

phetsims / aqua

Automatic QUality Assurance

MIT License

2 stars 4 forks source link

CTQ can't handle one really bad codebase change #197

Closed zepumph closed 9 months ago

zepumph commented 9 months ago

CTQ was not tolerant to the following horrible mistake I accidentally pushed before I was ready. (While working on https://github.com/phetsims/phet-io/issues/1965)

To reproduce: I renamed Client.ts to PhetioClient.ts without changing any usages. Then I pushed. So everything was broken.

CTQ kept saying: CTQ has caught unhandledRejection and will now exit.

Every problem our codebase could ever have should be caught, no matter how destructive, so I consider this a bug in CTQ.

zepumph commented 9 months ago

Ahh! This was a really important bug to grab. Basically anything that failed in puppeteer during load time wasn't getting caught. CT Quick made this shown easily, but likely most node clients were not correctly reporting load-time errors on CT. Uh oh. That is less than ideal. We'll check on CT tomorrow and see if there is anything sneaky going on. My guess is no, but likely the pageload tests haven't been getting to coverage for the last year.

zepumph commented 9 months ago

This is working much better now! I'm quite happy. I want to head over to https://github.com/phetsims/aqua/issues/198 now.