Investigate the use of aria-describedby for object responses in dynamic description

[zepumph]

Over in https://github.com/phetsims/ratio-and-proportion/issues/157#issuecomment-686773999, @terracoda had a great idea to use aria-describedby to accomplish dynamic description in application roles and custom interactions. I want to see how it works in a dynamic context both on focus and on value change. Hopefully this can be a new tool for our description toolbox, especially since aria-valuetext doesn't work as we want on application roles.

[zepumph]

Using this code snippet, I found that aria-describedby will update very well when changing often:

<!-- custom both hands interaction -->
<div id="both-hands" aria-role-description="movable" role="application" tabindex="0"
     aria-describedby="both-hands-description">Both Hands
</div>
<p or div id="both-hands-description">"description-for-both-hands"</p or div>

<script>

  const desc = document.getElementById( 'both-hands-description' );
  setInterval( () => {
    desc.innerText = 'extra cool' + Math.random().toFixed( 2 );
  }, 1000 );
</script>

A few interaction notes:

1. On focus this comes after than name/role, instead of before. Above we get "Both Hands, movable extra cool 0.12"
2. When you focus the application, and then wait on the component, just listening, the description builds up and becomes old very quickly. This case is where the description would update because of model stepping/animation, and not from user input.
3. Expanding on (2), when you press a key, it then takes the most recent description, getting "in sync" again. This is great news! Most of the places I want to use this will only change when on focus as a result of user input, so we should be able to depend on it to deliver the most recent description after a key press.

Tagging @terracoda and @jessegreenberg about this investigation. I am going to move over to https://github.com/phetsims/ratio-and-proportion/issues/173 and begin using this for a custom interaction in RaP. Feel free to just unassign, comment, and close if there is nothing else to discuss here.

[zepumph]

Oh, it may be nice to try to reproduce the above results on different platforms. I only tested with NVDA on windows.

[jessegreenberg]

In general I would expect aria-describedby to only be read when focus/virtual cursor first lands on "both-hands" element. Are you saying it gets read more frequently than that?

[zepumph]

Yes indeed! Please try out the above snippet by focusing on the application div, and then hearing the description changes read to you as the describedby element changes.

[zepumph]

Over in https://github.com/phetsims/ratio-and-proportion/issues/173#issuecomment-692867563 @terracoda found that aria-describedby didn't work with VoiceOver on focus or when it changed. I would like to create a small test case for use on my iPad to see if I can reproduce, and to see if there is a case where it does work (perhaps dependent on knit picky HTML).

Here is the PDOM that didn't work:

<div data-trail-id="16-17-306-312-307-341" id="16-17-306-312-307-341" data-focusable="false" class="pdom-sibling"
     tabindex="-1">
  <div data-trail-id="16-17-306-312-307-341-342" id="16-17-306-312-307-341-342" tabindex="0" data-focusable="true"
       class="pdom-sibling" aria-label="Both hands" role="application" aria-roledescription="movable"
       aria-describedby="16-17-306-312-307-341-343">
    Both hands
  </div>
  <p data-trail-id="16-17-306-312-307-341-343" id="16-17-306-312-307-341-343" data-focusable="false"
     class="pdom-sibling" tabindex="-1">
    hands somewhat close to each other
  </p>
</div>

[zepumph]

I created an isolated HTML snippet to try this out. The description for the application changes every second.

<div data-focusable="false">
  <div tabindex="0" data-focusable="true"
       aria-label="Both hands" role="application" aria-roledescription="movable"
       aria-describedby="description">
    Both hands
  </div>
  <p id="description" data-focusable="false"
     tabindex="-1">
    hands somewhat close to each other
  </p>
</div>

<script>

  const x = document.getElementById( 'description' );
  setInterval( () => {
    x.innerText = Math.random().toFixed( 2 ) + "some string";
  }, 1000 );

</script>

I witness very different behavior on three devices:

• Chrome + NVDA - on focus you hear the description, and whenever it changes, NVDA updates you with the current description. If you stay focused, there becomes a large build up of description and if the description changes never stops changing, NVDA will never catch up. If you press a key, then it will forget the building queue of messages and skip to the most recent description. This behavior may not be acceptable for use as a dynamic response to application input.
• Firefox + NVDA - You hear the descriptions on focus just once, and not when it changes. If you move the virtual cursor to the actual description, then it will be spoken each time it changes (just not when focusing on the application role).
• Safari + VoiceOver - You don't hear description on focus or on change. When I move the virtual cursor to the description I hear, "text field, is editing, [DESCRIPTION]". This whole string repeats for each description update.

From this level of support, I don't think that aria-description is a reasonable technology to depend on in our current sims. I would love to hear @terracoda and @jessegreenberg's opinion on this issue, and then we can take this outcome on put it to use in Ratio and Proportion

[zepumph]

aria-describedby has been removed from RaP. At this level of support, especially for VO + Safari, it seemed no worth it in conjunction with role="application".

See https://github.com/phetsims/ratio-and-proportion/issues/173#issuecomment-700293120 an d previous comments for discussion specific to RAP.

I'm going to close this issue. @terracoda please reopen if you have more thoughts.