Voicing on iPad does not work if initially turned on

[KatieWoe]

Found during https://github.com/phetsims/qa/issues/778 This issue was seen on iPadOS 15 and iPadOS 14, in both the link in the issue above, and in published simulations. If you use the ?voicingInitiallyEnabled parameter, voicing doesn't work initially. The panel appears, but no voicing occurs. You need to turn it off then on again. If turned on from the menu, it does work.

[zepumph]

Thanks for the report. I will need to test this. I'll mark this as blocking voicing until I understand it more. It could just be from the query parameter, or it could be caused by assumptions that are more important and foundation to the voicing code.

[jessegreenberg]

Safari is the platform that requires some form of user input to synchronously request the first use of SpeechSynthesis. That was one of the reasons announceImmediately was important. It would be great if Utterance Queue was smart enough to automatically use announceImmediately the first addToBack, maybe optionally.

[zepumph]

Ahh, good idea. Thanks!

[jessegreenberg]

Transferring to utterance-queue since that is where SpeechSynthesisAnnouncer lives now.

[jessegreenberg]

It looks like there was a hasSpoken flag in SpeechSynthesisAnnouncer doing something like this but it is no longer used. This kind of thing belongs in UtteranceQueue now.

This is a bit complicated because we can't just use announceImmediately the first time addToBack is used, it has to be used the first time addToBack is used from a listener on user input".

Potential options: 1) We use announceImmediately every addToBack until the first time synth.speaking is true. 2) We just do it the first time addToBack is used because usually addToBack happens in response to user input? No, can't do this. If the first addToBack is not from a user event and we call speakImmediately there the browser will still block that one and we will never try to speakImmediately again. 3) Just make the client use announceImmediately from a UI component when we want speech. voicingInitiallyEnabled won't work with safari. This is fine for Voicing but would put more burden on usages like the one in number-play.

[jessegreenberg]

Option 1 is working well, I tested on iOS Safari, macOS Safari and on Chrome. This is not an issue for Chrome but I saw that the first addToBack used announceImmediately when I commented out the "engine wake workaround" in SpeechSynthesisAnnouncer.

I added an option to Announcer.ts called announceImmediatelyUntilSpeaking that will keep trying to use announceImmediately until the synth is activated. I added it to Announcer instead of UtteranceQueue because it is an option we want for the speech synth tech being used and I wanted to be able to set this option to true for all SpeechSynthesisAnnouncers.

@zepumph would you mind reviewing this approach?

[zepumph]

Can you note why we needed an option to UtteranceQueue? Why not just announce immediately for all announcers? Is it buggy for AriaLive?

Otherwise things look good.

[jessegreenberg]

This felt like a platform specific workaround and I think I was trying to avoid applying it unless it was necessary. But I get what you are saying, and I think it could simplify the file if can just remove this. Ill do this and quickly spot check a few screen readers.

[jessegreenberg]

NVDA + Chrome - good NVDA + Firefox - good VoiceOver + Safari - good

OK, Ill commit to this.

[jessegreenberg]

Done, @zepumph would you like to spot check or anything else here?

[zepumph]

Yes I think that is really good. Thanks! I totally hear the thought that we don't want to make changes broadly to fix small, device-specific issues. That said, I also like keeping the code as simple as possible, and when it seems safe to do so, having just a single way of doing things. As long as there is good documentation (as there is in this case) noting why the code exists.