search

phetsims / aqua

Automatic QUality Assurance
MIT License
2 stars 4 forks source link

CT should run multitouch fuzzing without pan and zoom #117

Closed jessegreenberg closed 3 years ago

jessegreenberg commented 3 years ago

From slack, @pixelzoom mentioned that most of the time testing is done with ?fuzz&fuzzPointers=2, the sim is zoomed in. So the multitouch fuzzing on CT is not often not hitting many UI components.

We should have a fuzz test with multitouch without pan and zoom enabled.

We still want to test pan and zoom in some way. We could add a multitouch fuzz test to CT with pan and zoom enabled, but that may take up too much testing time on CT. Or we could run specific tests for pan and zoom in one (or a few) repos so that the feature is tested without running it on all. Or we can use the priority feature of listContinuousTests.js to prioritize non-pan and zoom fuzzing and only test it when CT has the time.

Adding to developer meeting label to check in on the load on CT, review how the priority feature works, and see if anyone has a preference.

jessegreenberg commented 3 years ago

Also, @pixelzoom requested that ?supportsPanAndZoom be added to phetmarks for easy access. Ill add that.

zepumph commented 3 years ago

@jessegreenberg it seems like writing unit tests that haven't been create to this point may be contrived if the best way to test is just fuzzing a sim to exercise pan/zoom. That said, if there are more tests that can be added for pan/zoom, then that makes sense to do in unit tests.

I adjusted the testing in a way that made sense to me, using the priority feature of CT to make sure that we didn't add too much fluff that delayed the more important load of CT.

I am constantly hesitant these days of adding an issue to dev meeting given the current load, so @jessegreenberg could you please review the above commit, and recommend if you would still like to set this for dev meeting.

jessegreenberg commented 3 years ago

Commits look good to me, and I see value in testing the pan and zoom feature on every sim. The feature should be general but there are sim-specific things that could come out of testing (https://github.com/phetsims/density/issues/56 is an example that came up recently).

The other question was about the priority option, and I see you added some doc about it, thanks.

I also checked in with @jonathanolson and he wasn't concerned about the burden this added to CT.

Closing.