Voicing missing important alerts

[Nancy-Salpepi]

Test device MacBook Air (m1 chip)

Operating System 13.1

Browser safari/chrome

Problem description originally mentioned in https://github.com/phetsims/friction/issues/322#issuecomment-1402018007

For https://github.com/phetsims/qa/issues/886, Voicing often skips over the statement "Jiggling more. Warmer" even though it shows up in the console with ?printVoicingResponses.
Also, sometimes when rubbing fast, I don't hear or see the alert, "Very hot. More atoms break away."

Steps to reproduce

1. Turn on Voicing and select all Voicing Options
2. Use keyboard nav to move book

Troubleshooting information: !!!!! DO NOT EDIT !!!!! Name: ‪Friction‬ URL: https://phet-dev.colorado.edu/html/friction/1.6.0-rc.1/phet/friction_all_phet.html Version: 1.6.0-rc.1 2023-01-17 16:32:22 UTC Features missing: applicationcache, applicationcache, touch Flags: pixelRatioScaling User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/109.0.0.0 Safari/537.36 Language: en-US Window: 1536x781 Pixel Ratio: 1.7999999523162842/1 WebGL: WebGL 1.0 (OpenGL ES 2.0 Chromium) GLSL: WebGL GLSL ES 1.0 (OpenGL ES GLSL ES 1.0 Chromium) Vendor: WebKit (WebKit WebGL) Vertex: attribs: 16 varying: 31 uniform: 1024 Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32) Max viewport: 16384x16384 OES_texture_float: true Dependencies JSON: {}

[Nancy-Salpepi]

Based on conversions with @jessegreenberg and @zepumph I tried using only quick single key strokes. I can hear all the alerts this way, but I don't picture students that have vision and are using voicing to be using the sim in this way.

[jessegreenberg]

From looking at the code the only high priority utterances are the ones that describe particles breaking away. I think all others (including those describing "warmer") may be interrupted by other utterances, even if they show up in the dev tools.

If I leave a bit of time between causing particles to break away I hear every shearing alert. I suspect they are getting missed because of this timeout: https://github.com/phetsims/friction/blob/56d8e713ea311670d3fece7c2dc89ca6175f503f/js/friction/view/BreakAwayAlerter.js#L106

I could also see tooSoonForNextAlert may prevent hearing when there are no particles left? But I am not too sure, we can review next week.

[zepumph]

@jessegreenberg and I found a case where the break-away alert was being cleared from the queue. @Nancy-Salpepi can you please take a look at master and see if things sound better. At this point you really should be hearing the break away alert over every single thing.

[Nancy-Salpepi]

Sounding better in master. I definitely hear the break away alert each time now 🙂. Often times I do hear "Jiggling even less. Even cooler" right after the break away alert. It is a little odd to move that quickly from "Very hot" to "Even cooler."

Overall though, I think that users can reach the correct conclusions about what is happening at the molecular level with Voicing enabled.

[zepumph]

Often times I do hear "Jiggling even less. Even cooler" right after the break away alert. It is a little odd to move that quickly from "Very hot" to "Even cooler."

@Nancy-Salpepi, that is part of the tradeoffs built into the design. We have no way of getting the "end of speaking" event with aria-live. This sim was developed with description first, and then voicing was added after the fact, and to try to match the implementation with description rather than implement a totally new design. Because of this, the timing is based on empirically calculated timeouts to help guide the use down an "average" case. In general I'd say it works "ok" and not better than that, but we learned a lot while doing this and have applied that learning to Quad and Greenhouse for better "ambient description" implementations.

We are ready got cherry-pick

[jessegreenberg]

cherry-pick done, I briefly made sure Voicing still sounds good while breaking away particles. Ready to verify.

[Nancy-Salpepi]

Sounds good in rc.3. I definitely hear the break away alerts 🙂. Closing.