Investigate/consider adding support for voice control software (such as NaturallySpeaking)

[arouinfar]

We recently did user testing on the Waves screen of Greenhouse Effect using Fable. One cohort consisted of users who use some form of assistive technology (excluding screen readers). Two of the participants in that cohort use a voice control software, Dragon NaturallySpeaking, and reported significant issues. Here are two quotes from the report:

"I noticed that none of the areas on the site were labelled as link options when I used the command "show links" in Dragon NaturallySpeaking. I woulD recommend labelling each clickable area as a link to make navigating the website easier. The only way I could click into certain areas was if I used the MouseGrid functionality within my dictation program."

"Nothing on this page is labelled according to Dragon. When I use the command "show links" nothing on the screen is numbered for Dragon to select. So the only way I can select anything on this page would be with the "mouse grid" command. The MouseGrid method is usually the last resort I would use to navigate due to it needing many commands just to select something."

The Greenhouse Effect team was not sure if we have ever investigated supporting voice control software, or if there are standards we could follow to improve access for these users, but I wanted to make sure to pass this information along.

[terracoda]

We have not tested with Dragon users, so it is wonderful to have this opportunity and to get feedback from some.

I don’t know the strategies that Dragon users typically use, I would guess that controls without visible labels/names would be challenging.

Typical websites have links. Our sims don’t have links, so I am confused by that strategy.

I can only hazard a guess that show buttons or show labels (if valid Dragon commands) would be a better strategy for our sims.

I totally agree that the mouse grid is a last resort navigation option and would not provide a good user experience.

All our interactive controls have accessible names, but those names are not always visually displayed, so a Dragon user would have to guess at the names or know the name in advance.

My understanding of Dragon is limited , but I think/thought that it will find accessible names if the name is spoken out loud.

If a Dragon user is aware that the PDOM exists, and that it has headings, descriptions and labels for controls, they might use a different strategy.

So happy to hear about this testing. Now would be a good time to learn about Dragon’s speech commands to understand what strategies work best with our sims or if we need to add anything to the code.

[terracoda]

Here’s a blog post about dragon commands and accessibility testing https://www.barrierbreak.com/cheat-sheet-dragon-naturally-speaking-commands-for-web-testing/

It’s nice that there keyboard like commands like Tab (n) times.

“Tab 1 time” should get a dragon user to the Sunlight button, for example.

[terracoda]

@arouinfar, I read the UX report a bit more and there might be a very interesting nuance here between Dragon keyboard-type commands and Dragon mouse-type commands. The accessible names only exist in the PDOM, so I am wondering if Dragon mouse-type commands can even be made to work at all. I think Dragon Tab navigation strategies should work, but if one does not Tab to the Start Sunlight button, and only tries to click it by name it might not work. The button’s name is in the PDOM not in the clickable visible button.

[emily-phet]

@arouinfar i think this would require some further investigation to sort out what is currently the most effective access, what might be possible, and what (if any) implementation would be involved. Let us know if you (or whoever would be doing so) would like support in that investigation.