(Interactive Description) Grab Yellow balloon button not communicating name and role

[terracoda]

When reviewing #490 and #491, I noticed that sometimes I hear "article web application" when putting my focus on the grab yellow balloon and grab green balloon buttons.

This seems to happen after a balloon has been grabbed and then released. Pressing the rest balloon button or reset all does not reset the button back to "Grab {{Yellow/Green}} Balloon". Reloading the page put the proper label back on the button.

@jessegreenberg, I am wondering if the button is not coming back into the code properly (not being unhidden or something like that) when the balloons are release because, but I am not sure exactly what's happening because I can still grab it and move it.

On first focus I hear

• "Grab Yellow Balloon, button
• When I grab it and release it, I think I am hearing everything I am supposed to hear.
• When I move focus and then refocus I hear, "Web Application"
• If I leave my focus on the balloon go to another browser tab and then come back to the the tab with BASE, VoiceOver says, "Yellow Balloon" with the role button, then if I move focus and refocus, VO says again "Web Application".

I'm thinking it has to be something to do with the roles in the code.

[jessegreenberg]

Here is the HTML for the balloon on load:

here is the HTML when it is grabbed:

And when it is released:

The role information in these images looks correct.

Doing the same steps on Firefox with NVDA I hear

• First focus: "Grab Yellow Balloon, button"
• First pickup: "Yellow Balloon, movable, grabbed, ...[other description]"
• Release with spacebar: "Released. Grab yellow balloon, button"
• Tab to Reset All Button: "Reset All, button"
• Tab back to Yellow Balloon: "Grab Yellow Balloon, button"

I will try VoiceOver.

[jessegreenberg]

I am able to produce on VoiceOver easily. I also notice in addition to the loss of name and role that the black VoiceOver highlight changes for the balloon to a small black rectangle in its center. Not sure if that is related, but it is strange.

Copying more of the Balloon HTML to see if something about ancestor HTML is changing and causing this: On load:

When grabbed:

When released:

[jessegreenberg]

Here are the entries in the accessibility Tree for the button after it has been grabbed:

Looks correct.

This is not an issue for the published version, here is the output there:

THey look the same. Note these were takin in Windows Chrome, there is no such tool in Safari.

[jessegreenberg]

I am seeing this in other sims too, not specific to BASE.

[jessegreenberg]

In the published version, there were two DOM elements, one for the button and one for the draggable balloon. Focus moved between them when changing state. GrabDRagInteraction uses one element for everything, creating and destroying new ones depending on state. That could explain why this is not an issue in the published version.

[jessegreenberg]

@terracoda and I are both using Big Sur 11.4 and seeing this on master. @terracoda said this is not happening in High Sierra 10.13.6.

[jessegreenberg]

@terracoda had a thought to see if this is happening in Firefox on macOS. That would help us determine if this is a VoiceOver issue or a Firefox issue.

[jessegreenberg]

This is NOT happening in macOS 11.4 Firefox 89.0.2, confirming that this is a Safari bug.

[jessegreenberg]

Discussed with @emily-phet and @terracoda:

@terracoda suggedsted maybe try an aria-label in addition to the inner content of the button. Or adding aria-labelledby to force the screen reader to say something about the button.

Also, we could try to use the implementation for the grab/drag pattern that was used last time BASE was published, which involved two persistent HTML elements. Before changing common code, lets try a basic HTML example before modifying common code. If we think it will fix it, then we can make changes to GrabDragInteraction.

@emily-phet and @terracoda confirmed that this is fundamental to the sim interaction and should be fixed.

[jessegreenberg]

This should block publication. But it also involves the grab drag interaction so moving this to scenery-phet.

[jessegreenberg]

I started https://github.com/phetsims/scenery-phet/issues/688.

[jessegreenberg]

A fix was committed in https://github.com/phetsims/scenery-phet/issues/688 and it is sounding much better, closing this issue.