Investigate and discuss ARIA 1.1 changes relevant for sim interactions

[terracoda]

Issue to discuss ARIA 1.1 additions and changes that might be helpful for us:

For example,

New properties aria-roledescription, aria-keyshortcuts, aria-placeholder, aria-current , aria-modal.

I am especially interested in aria-roledescription. I am thinking we can use this property to replace the screen reader utterance, "Application" with something more meaningful like, "Yellow Balloon Grabbed," as the user activates the application role upon pressing the "Grab Yellow Balloon Button".

There's a handy Change Log for ongoing changes to the new specification.

[terracoda]

@emily-phet, @jessegreenberg, @zepumph and others.

FYI, I used aria-roledescription attribute in Home Screen and Navbar mockups in issue #45, and it works beautifully in Voice Over. Each sim screen button is read out as "Sim Screen" rather than, "button."

• Atom, Sim Screen
• Symbol, Sim Screen
• Game, Sim Screen

This new aria attribute should be extremely useful for PhETSims if well supported by other browsers and screen readers.

[jessegreenberg]

This looks awesome @terracoda, do you think we should test it on Firefox + NVDA and JAWS + Firefox/IE11 to see if it is supported on those platforms?

[jessegreenberg]

@terracoda do you have a link to an HTML version to test the attributes? Or can I pull from github? If so I have easy access to NVDA and JAWS and can test myself.

[terracoda]

@jessegreenberg, yes we should test. I meant to say that explicitly in my last comment.

Links to current HTML mock-ups are in issue #45 and the code lives in this repo in the html-sketches/sim-screens folder:

• Home Screen
• Atom Screen

[jessegreenberg]

Great, thanks! Ill take em for a test drive.

[terracoda]

@jessegreenberg, keep in mind the description content and the structure in the mock-ups is not final, so make note of awkward description-structure screen reader utterances. I was surprised how nice it sounded in Voice Over.

[jessegreenberg]

Aye - ARIA 1.1 be uncharted waters. I just want to test that roledescription comes through for NVDA and JAWS.

But if so, roledescription would be incredibly useful for many sims.

[jessegreenberg]

@terracoda do you know if/how roledescription might change keyboard interactions? For instance, could we have roledescription="Balloon" so we no longer need role="application"?

[terracoda]

@jessegreenberg, I think it is ok that it is ARIA 1.1. Voice Over with Safari already supports aria-roledescription correctly. I am assuming that if the AT/Browser does not support it, then AT with just announce the normal regular role, e.g. "button" in this case.

I have been thinking about the balloon case. aria-roledescription replaces the utterance of the normally read out role. The element must have a role in order to use aria-roledescription. In theory, it should work with role application as well as any other valid role, like button, or region, or whatever!

[terracoda]

@jessegreenberg, and aria-roledescription does not change keyboard interactions. It describes more specifically what the role's actual representation is.

A region could be a "slide" in a presentation for example.

[terracoda]

@jessegreenberg, shall we try the following change to Balloons and Static Electricity?

• Relabel the grab buttons to "Grab" only.
• Provide aria-roledescription="Yellow Balloon" for the Yellow balloon's grab button, and
• aria-roledescription="Green Balloon" for the Green balloon's grab button.

My guess (for Voice Over) is it would read the button out like this:

• "Grab, Yellow balloon," and then provide the following VO help text,
• "You are currently in web content on a Yellow Balloon. To click this Yellow Balloon press Control Option Space."

Would it be better to start an issue for this idea in the BASE repo? Of course, we would have to test other screen readers to see how they handle this aria-roledescription attribute because without it, the grab buttons would be identical.

[jessegreenberg]

Sounds interesting to me @terracoda! Then we don't have to hear "application" frequently. Yes an issue in BASE sounds great.

[terracoda]

@jessegreenberg, actually, in my example in https://github.com/phetsims/a11y-research/issues/6#issuecomment-318081399, the aria-roledescription is replacing the button role on the grab button. It might make better sense to use aria-roledescription once the balloon is grabbed, or perhaps in both cases, although the specification says to use it sparingly :-P

Here's the VO text that is provided after "Grab Yellow balloon, button": Providing the description here in the button would do what I explained in the above comment.

<button aria-roledescription="Yellow balloon">Grab</button>

Here's a screenshot of the grab alert once the balloon is grabbed:

I'll have to think about this a little, but maybe "Grabbed yellow balloon" is a good description? Of course, not sure if it fits so nicely with the way other screen readers handle this attribute.

[jessegreenberg]

@terracoda yes, wouldn't it be used on the div with role=application? Why would we want to change "Grab Yellow Balloon, button"?

[jessegreenberg]

In http://htmlpreview.github.io/?https://github.com/phetsims/a11y-research/blob/master/html-sketches/sim-screens/index.html, NVDA is ignoring aria-roledescription on the buttons, all I hear is "Atom, button" and so on.

[terracoda]

@jessegreenberg, since I was just playing with sim screen buttons, I was thinking about buttons, but I totally agree that the role application is a more desirable thing to replace.

[terracoda]

Thanks for testing in NVDA (https://github.com/phetsims/a11y-research/issues/6#issuecomment-318115617). This helps focus the issue for BASE. We don't want to remove any clear label text, in case AT does not support the attribute.

[jessegreenberg]

aria-roledescription works great with JAWS + IE11, but does not work with JAWS + Firefox.

[terracoda]

@jessegreenberg, I agree we need to think about how and when to add to a control's name with aria-roledescription. I am just gathering some examples here:

• Specification
• Example for slides

Wondering if using it on both the grab button and the application div would be useful for Balloons and Static Electricity.

Current screenshot of PDOM:

For example:

<div tabindex="-1" id="container-div-id">
  <button id="button-id" aria-roledescription="grab button" tabindex="0">Yellow Balloon</button>
  <div tabindex="-1" id="yellow-balloon-id" role="application" aria-roledescription="grabbed" hidden>Yellow Balloon</div> 
</div>

A screen reader user may hear something like: "Yellow Balloon, grab button" and then once the Yellow Balloon button is activated, they might hear, "Yellow balloon, grabbed" and then the interactive alert that describes position and charge.

On the button, Voice Over help text might sound like this: You are currently on a Grab button (instead of "button"), inside of web content. To click this Grab button, press Control-Option-Space. To exit this web area, press Control-Option-Shift-Up Arrow

In the grabbed state, Voice Over help text might sound like this: "You are currently on a Yellow Balloon (instead of "text element") inside of web content. To exit this web area press, Control-Option-Shift Up Arrow."

On release, the button regains focus, and the user would likely hear the button's label and description again, "Yellow Balloon, grab button". This may be an improvement from the current label text, "Grab Yellow Balloon."

Of course, not all screen readers support aria-roledescription, so I am just exploring what Voice Over does in this issue. VO has supported it since about 2015.

[terracoda]

As discussed in the A11y Design meeting. I t would be good to think about how aria-roledescription could be used to support a direct custom interaction. The goal would be to eliminate the use of the button (a native interaction which is very accessible to screen reader users using the keyboard) and make the custom interaction easily understandable by everyone (visual keyboard, touch users, screen reader users, mouse users, switch users, etc.).

[zepumph]

make the custom interaction easily understandable by everyone (visual keyboard, touch users, screen reader users, mouse users, switch users, etc.).

I'm a bit confused by this, but I'm sure you can cure case. I thought there is a push in a11y to try to pass as much semantic information to the use as possible. Isn't using an html <button> one of the best ways to communicate that you are 'clickable' or whatnot? Are a11y switch or touch users less used to seeing buttons, and so would benefit from a more custom approach? Is it possible to just add a custom gesture interaction to the <button>?

[terracoda]

@zepumph, the button creates native "grab interaction" that is easily understood by many, and easily operated by many. It's an issue for visual keyboard users because the balloon does not look like a button (so we created a visual hint for these users). It is problematic for touch users (as discussed today).

The reason we created the Grab button was to specifically deal with confusion caused by verbosity of the a custom element created with the application role. The application role has changed significantly (at least how it sounds in VO has changed), so it might be worth exploring if the balloon can be easily grabbed by screen reader users without the use of a button. I'm not sure this would solve all our design issues for all users. But it might allow for a more direct approach for all users?

[terracoda]

Not sure if describing the state of the application together with an associated action (all dynamically) would work. The only examples I have seen are very simple (slide on section element, attach something for button).

For example, what if there were two states for the balloon application, grabbed and released. When the state is grabbed the associated action is described as "move" and when the state of is released, the associated action is described as "grab".

<div tabindex="-1" id="container-div-id">
  <div tabindex="-1" id="yellow-balloon-id" role="application" aria-roledescription="Released balloon, grab me." hidden>Yellow Balloon</div> 
</div>

Once grabbed, the aria-roedescription changes:

<div tabindex="-1" id="container-div-id">
  <div tabindex="-1" id="yellow-balloon-id" role="application" aria-roledescription="Grabbed balloon, move me." hidden>Yellow Balloon</div> 
</div>

[terracoda]

@jessegreenberg and @zepumph, that's all my thoughts on this today. The interaction for grabbing would still be a two-step process, so not sure how much we gain. However, it would be a two-step process with a dynamic description which potentially could be helpful if it is possible.

[jessegreenberg]

In https://github.com/phetsims/a11y-research/issues/58 I found that aria-roledescription is supported in Safari + VO, but not with FF with JAWS and NVDA, so the value of aria-roledescription will only be read by by VO users.

[jessegreenberg]

Is it possible to just add a custom gesture interaction to the

I don't think that is possible for web apps, gestures are enabled by the browser/assistive technology and depend on what has focus. For buttons, the gesture is to double tap the screen to activate, and all other gesture doesn't even create an event for the browser to catch.

[terracoda]

@jessegreenberg with little support, I agree we cannot move forward with any critical information being provided solely by aria-roledescription.

[terracoda]

@jessegreenberg and @zepumph, I also found some discussion about what people think aria-roledescription should be allowed to do from about a year ago.

https://github.com/w3c/aria/issues/500

I like the hypothetical "pizza pie" example. Seems a bit more like our situation.

[terracoda]

Just curious, what is the protocol on referencing repos outside one's organization?

[zepumph]

That is a good question @terracoda. I have referenced outside issues before also, though mainly accidentally. @ariel-phet do you have an opinion? Has this come up before?

[terracoda]

Yes, I have done this twice now with WAI ARIA documentation (first-time by accident, too), so I thought I should ask.

[ariel-phet]

@terracoda thanks for asking. In general referencing outside repos is totally fine. We often reference outside repos/issues/websites etc.

[zepumph]

Unfortunately for this issue, aria 1.1. still has pretty minimal support across browsers and AT. I went to a talk at CSUN two months ago about this. The slides are below, they have support charts for a few key aria roles and attributes. csun2018-aria11-newShiny-sdamera-dboudreau-20180321-final.pdf

[terracoda]

@zepumph thanks for the slides for that talk. I coudn't get in as it was full.

Out if the 10 attributes discusses, the ones most relevant to us are:

• switch
• aria-current
• aria-haspopup
• aria-roledescription

I think we have also tried these 4 out in sims and know we cannot rely on them yet.

I always check ARIA Practices 1.1 for the latest examples but then refine and test based on our context. I also often look for other working examples.

I think this seems like a good flow.