phetsims / friction

"Friction" is an educational simulation in HTML5, by PhET Interactive Simulations.

http://phet.colorado.edu/en/simulation/friction

GNU General Public License v3.0

4 stars 6 forks source link

VO repeats descriptions in Wrappers #286

Closed Nancy-Salpepi closed 1 year ago

Nancy-Salpepi commented 2 years ago

Test device MacBook Air (m1 chip)

Operating System 12.3

Browser Safari

Problem description For https://github.com/phetsims/qa/issues/791, VoiceOver repeats descriptions in the downstream sim. This only happens for changes made in the sim (moving the book, moving a toggle button on/off).

Steps to reproduce

In the state wrapper, set the Set State Rate = 0
Make a change in the upstream sim and press 'Set State Now'
Turn on VO and tab into the play area of the downstream sim
Move book back and forth -- descriptions will repeat

Visuals

https://user-images.githubusercontent.com/87318828/161258964-705f1464-3cca-4dad-adc6-a53bcfb3b305.mov

Nancy-Salpepi commented 2 years ago

Also seen in the following wrappers: Data: textarea, Data: recording, Screenshot, Multi, and Active wrappers (unable to test studio at this time)

DevonQui commented 2 years ago

I tested these cases out on an M1 Macbook as well and saw the same behavior where VoiceOver repeated descriptions when changes were made to the sim.

terracoda commented 2 years ago

Just and FYI, when we used to do Sound design using a wrapper, this type of repetition was very common. I do not know what causes it.

zepumph commented 1 year ago

I couldn't reproduce this with windows/chrome/nvda. I'll need to solicit a mac user.

jessegreenberg commented 1 year ago

I was able to reproduce this problem with MacOS 11.6.2 Safari. I tried a basic HTML + js example but could not get this to happen.

index:

```html

Hello world! This is HTML5 Boilerplate.

```

content:

```html Title

Example iFrame content

```

I tried

aria-live="polite" and aria-live="assertive" to see if one caused it.
Using the platform workarounds we have in AriaLiveAlerter in our example, so my content.html looks like

```html Title

Example iFrame content

```

I am not sure why this is happening in Safari in iFrames but I am not convinced yet that it is not our fault. But I am not sure if we should spend more time on this.

Outside of PhET-iO, I know that embedding PhET sims in iFrames is a popular use case so I am sad that this is broken.

Also, the experience with the sim on my Mac was very poor. VoiceOver almost always just said "Safari busy..." as it struggled with performance. If we continue here we may want to test by remote access into a different PhET machine.

zepumph commented 1 year ago

@jessegreenberg and I discussed this a bit more. We don't see any potential workarounds, and we don't think it is worth spending more time reproducing the issue at this time. This does not block Friction.

zepumph commented 1 year ago

@terracoda, @emily-phet, and I think we should create a bug report for this. If any of our third-party embedder of PhET sims reach out with this issue, we can recommend directly using the sim from our website (a bad workaround, but potentially possible).

UPDATE: @jessegreenberg helped me realize that I read https://github.com/phetsims/friction/issues/286#issuecomment-1185982974 incorrectly. It isn't reproducible in that snippet. We will spend a modest amount of time seeing if it is possible (give our moderate success and large amount of fun doing the same thing last week in https://github.com/phetsims/a11y-research/issues/175). I'll report back.

jessegreenberg commented 1 year ago

We came back to this today and tried not to @jessegreenberg on a Mac 11.6.2 - Could not reproduce this (tested on https://phet-dev.colorado.edu/html/john-travoltage/1.6.4/phet/john-travoltage_all_iframe_phet.html) @chrisklus on a Mac 10.13.6 Safari 13.1 - Tested (https://phet-dev.colorado.edu/html/john-travoltage/1.6.4/phet/john-travoltage_all_iframe_phet.html) @Nancy-Salpepi on a Mac 12.5.1, Safar 15.6.1 - It happened immediately in both RAP and JT

Then, @jessegreenberg tried again DID see the problem happen in john-travoltage example. We created some more examples to test on @Nancy-Salpepi machine to see if it will happen outside of PhET. If so we can either identify and fix the problem or submit a bug report to Apple. See examples here: https://phet-dev.colorado.edu/html/jg-tests/iframe-tests/

jessegreenberg commented 1 year ago

We got it to happen on my machine in the most basic case by doing the following:

Go to https://phet-dev.colorado.edu/html/jg-tests/iframe-tests/basic/.
Enter the iframe with the virtual cursor and read the first heading in the content.
If the aria-live element changes while you are reading the heading with a virtual cursor you will hear the alert content twice.

jessegreenberg commented 1 year ago

This is about as simple as it gets for aria-live we are ready to submit this example to Apple as a bug report. Since it is outside of our hands we will also make sure it is in the list of known platform issues.

zepumph commented 1 year ago

Thanks so much to @jessegreenberg @Nancy-Salpepi and @chrisklus for helping to investigate this bug. We couldn't have done it without you all. I'm going to close this issue as a won'tfix knowing that an apple bug report will be given for https://github.com/phetsims/a11y-research/issues/177.

Closing

Nancy-Salpepi commented 1 year ago

I know this was closed as "won't fix," but wanted to note that I also see repetition in the Standard Phet-iO wrapper in the most recent dev test (https://github.com/phetsims/qa/issues/868).

EDIT: I also see this bug was added to the list of known a11y bugs. Thanks 🙂