Support for Talkback

[jessegreenberg]

We don't currently support talkback yet (Google screen reader for android), but we would like to in the future.

We tried some preliminary testing in https://github.com/phetsims/QA/issues/333#issuecomment-498790543, but it wasn't a full test and we should do more soon. I will create a QA task and coordinate with @KatieWoe to test on an up to date android device.

[jessegreenberg]

I notice a few issues on my Galaxy S6 with Talkback on Chrome 75. aria-valuetext is not supported at all which is a problem because we use aria-valuetext a lot in our sims. Whenever I click on a button, Talkback says "Off" as if every button is a toggle button.

[jessegreenberg]

In https://github.com/phetsims/tambo/issues/65 we found that sound does not initiate from input or change events on Galaxy S6 with Talkback on Chrome 75. This means that there are cases where the user could interact with a simulation and not hear any sound. For instance, if the first thing a user interacts with is a slider, any associated sound will not come through.

[jessegreenberg]

To be continued after https://github.com/phetsims/QA/issues/364

[jessegreenberg]

@KatieWoe and I tested Talkback in https://github.com/phetsims/QA/issues/364#issuecomment-510934500.

A summary of what we found:

• Using gestures to read through content with the cursor works great.
• You cannot tap individual elements in the sim to focus them. This is specific to PhET sims, this is possible for Talkback with other web content.
• Interactive content is not discoverable if reading setting is set to "lines" - it must be "default" or something else.
• Sliders are adjusted with volume keys once focused (this is different than VoiceOver)
• Talkback does not use DOM focus when reading with the cursor, so Talkback will have the similar problems as https://github.com/phetsims/a11y-research/issues/138. However, DOM focus is placed on an element once it is activated. So the PhET pink highlight will generally show up on elements once they are clicked. This also means that focus listeners would be called on activation which might produce unexpected results since this is so different from desktop AT.
• aria-valuetext is not working on Talkback. Instead, talkback reads the percentage of the value relative to the slider range. This is a big problem, we convey a lot of info through aria-valutetext.
• aria-live alerts are working great.
• buttons and checkboxes are working great.
• aria-pressed is working great, the "Sound" toggle button state was conveyed.
• When you first encounter a slider, the label and accessible name is read but you can't interact with it yet. You must navigate further to the input element to be able to adjust the slider.
• Sound does not initiate from input or change events on Galaxy S6 with Talkback on Chrome 75. This means that there are cases where the user could interact with a simulation and not hear any sound.

The biggest issues are no support for aria-valuetext, and you cannot tap to find elements on screen (important for low vision users).

We should determine if aria-valuetext is supported outside of PhET. The tapping issue is PhET specific but may not be solvable with our current solution for mobile a11y.

[jessegreenberg]

At this JSFiddle, my Galaxy S6 device is reading the aria-valuetext after the slider percentage. https://jsfiddle.net/bp4akf2r/6/

EDIT: I discovered that simply having a label element for a slider prevents the aria-valuetext from being read, which is why it isn't happening in our sims. https://jsfiddle.net/oz9Lpyce/

[jessegreenberg]

After playing with CSS in AccessibleSiblingStyle, I found that these two CSS attributes are preventing touch discovery on talkback:

      // 'pointer-events: none;' +
      // 'clip: rect(1px, 1px, 1px, 1px);' +

[jessegreenberg]

We are ready to have a group discussion about whether to push for support for Talkback.

[jessegreenberg]

We discussed this as a group and determined that Talkback does not work well enough to push for broad support. The biggest drawback is the lack of support for aria-valuetext. This indicates there will be other important attributes that are not supported. We will continue to hope that Talkback improves and continue to check on it about once per a11y related dev test to see if support gets better. We will add a section to the a11y dev task to ask if Talkback should be checked as part of the test to make sure we keep up with this.