search

phetsims / scenery

Scenery is an HTML5 scene graph.
MIT License
52 stars 12 forks source link

Voicing needs better CT coverage #1338

Open zepumph opened 2 years ago

zepumph commented 2 years ago

From https://github.com/phetsims/scenery/issues/1337. To me it is a bit crazy that that bug wasn't getting caught on CT, but it is because voicing needs to be enabled to get things working. This is also related to https://github.com/phetsims/scenery/issues/1264.

I think the best path forward is to just add some tests for voicing in listContinuousTests

zepumph commented 2 years ago

I added a voicing list, and added tests for it. @jessegreenberg can you please review.

jessegreenberg commented 2 years ago

This will be helpful, thank you. I also see entries now in the new data/voicing list. And according to continuous-testing-management.md changes to listContinuousTests.js will be included in the next snapshot.

Closing.

jessegreenberg commented 2 years ago

While reviewing today @zepumph noticed that tests aren't running on CT, we should figure out why.

jessegreenberg commented 2 years ago

It seems that tests are running but we were surprised that sims were running without error (we expected to see github.com/phetsims/utterance-queue/issues/46).

We found that headless chrome does not support speech synthesis (https://github.com/puppeteer/puppeteer/issues/2141#issuecomment-370525613). Also since there is no legitimate input with fuzzing SpeechSynthesis will generally be disabled in tests.

I am not sure how to get around this.

zepumph commented 2 years ago

A bit of background. . . Here are the passing voicing tests from the past night:

image

But note that because of https://github.com/phetsims/utterance-queue/issues/46, pretty much all repetitive voicing responses are broken right now. For example just drag any hand in ratio and proportion in this link and it breaks (or with <10 seconds of fuzzing after you click into the window):

https://phet-dev.colorado.edu/html/ratio-and-proportion/1.2.0-dev.4/phet/ratio-and-proportion_all_phet.html?voicingInitiallyEnabled

https://phet-dev.colorado.edu/html/ratio-and-proportion/1.2.0-dev.4/phet/ratio-and-proportion_all_phet.html?voicingInitiallyEnabled&fuzz

I'd like to discuss in dev meeting, since this applies to sound as well.

jessegreenberg commented 2 years ago

We brainstormed this in dev meeting this week. One thought was to make the behavior separate from the speech output so that this is not an issue in the fist place. Currently the behavior is tightly coupled with browser information/events that let us know what is being spoken and when. It could be quite difficult to change that. But that may be easier than creating new testing tools.

It is a bummer that the voicingUtteranceQueue is essentially not getting tested on CT, but I don't think I can explore this any further now. @zepumph how would you like to proceed with this issue?