How well are Arrow keys working in JAWS for 4-way custom move?

[terracoda]

I am having some trouble assessing what is the best way to cue the custom 4-way drag, especially for Friction where pressing the W key is not so useful.

In a recent session with a VoiceOver user familiar with our sims, there seemed to be confusion around cuing only the use of the WASD keys for moving the book.

Goal of this issue: Assess how well the Arrow keys are working in JAWS so we can figure out the best way to provide instructions for the interaction and the best way to provide alternatives for the interaction.

Question 1 for @jessegreenberg Do you know how well the Arrow keys are working with JAWS now? In early prototypes of BASE the Arrow keys were not working at all with JAWS, and then they worked, but not as well as the WASD keys.

Question 2 for @terracoda & @emily-phet If JAWS is no longer having a serious problem with releasing the Arrow keys to the web application, how and where is the best way to provide the alternative WASD key instructions?

• In PDOM help text, first grab alert, AND Keyboard Shortcuts dialog?
• Only in PDOM help text AND Keyboard Shortcuts? OR
• Only in the Keyboard Shortcuts dialog?

Currently, we do option 1, and we only explicitly refer to using the Arrow keys in the Keyboard Shortcuts dialog. Here are examples from BASE and Friction, and soon there will be a grab buttons in GFL and Faraday's Law.

Cuing text for BASE

• PDOM help text:

  • "Look for grab button to play. Once grabbed, press W, A, S, or D key to move up, left, down, or right. Space to release."

• Initial Grab Alert:

  • "Grabbed. At center of Play Area. Has no more negative charges than positive charges. Press W, A, S, or D key to move balloon. Space to release."

• Keyboard Dialog content for grabbed balloon:

  • "Move grabbed balloon up, left, down, or right with Arrow keys or with letter keys W, A, S, or D."

Cuing text for Friction

• PDOM help text:

  • "Look for grab buttons. Once grabbed, use letter keys W, A, S, or D to move book or zoomed-in book up, left, down, or right."

• Initial Grab Alerts before successful interaction:

  • Not touching: "Grabbed. Lightly on Physics book. Use W, A, S, or D keys to move book. Space to release."
  • Touching: "Grabbed. Rub fast or slow with A or D keys. Space to release."
  • and immediately after a grab alert the user also hears, "Atoms jiggle a tiny bit, temperature cool"

Keyboard Dialog content Friction's grabbed book:

• "Move grabbed book up, left, down, or right with Arrow keys, or with letter keys W, A, S, or D."

[jessegreenberg]

@terracoda I just tested with JAWS 2019 in both Chrome and Firefox. Unfortunately the arrow keys still do not work with JAWS because of https://github.com/phetsims/a11y-research/issues/9.

[terracoda]

Considering that JAWS still does not work with the Arrow keys, I think the WASD letter cue should be in all three places, but I could be convinced of only in PDOM hint and Keyboard Shortcuts dialog if a generic directional instructions are used in the INITIAL alert - no reference to arrow or WASD keys.

Doing this might mean a couple of failed attempts for JAWS users?

[terracoda]

@emily-phet, please add any thoughts or concerns you have. I am thinking about the cues in Friction as I wrote this.

[terracoda]

I'll mark this as meeting topic. We can discuss today without @emily-phet or next week (Nov 27) if that is better.

[terracoda]

@emily-phet, we didn't get to this topic on Nov 20. We can potentially discuss Nov 27. In the mean time I am forging ahead for Grab cues for Friction's book and GFL's ruler.

[emily-phet]

My initial thoughts:

• I think it isn't ideal to have a generic movement cue on grab (that doesn't include arrow keys or WASD keys explicitly). I think those less familiar with interactive game-like laptop experiences, may start out a bit lost as to what to do. Since we know that users tend to just cycle around to other - less important - objects on screen when confused about a grab/move interaction, it's very important we get them successfully interacting as quickly as possible, even if that means risking some verbosity. Since all users should be able to use the WASD keys, I suggest we stick with WASD keys being part of the alert on grab. (and in all the other places as well)
• For Friction, it's possible to successfully use the sim and never use the 'up' arrow key/'w' key. Users can press down, and then reset to get the book back up higher. You might consider just having a custom cue that only indicates ASD keys to go left/down/right. Or front with "S" key to go down, A/D to go left/right - to help indicate the relative importance of each movement. This might also allow you to add in something specific to A/D (as in, move left/right fast or slow) or something like that, if it seems helpful.

[terracoda]

@emily-phet, I agree, thus far there are interaction nuances in different sims that likely prevent a general alert without explicit key press instructions.

So I also think that until JAWS releases the arrow keys with role application we should provide explicit instructions in three consistent places:

1. The PDOM help text near the actual custom object.
2. In the initial grab alert (potentially a little briefer than the help text in the PDOM).
3. In the Keyboard Shortcuts dialog.

[terracoda]

Discussed at A11y Dev meeting, and we will only cue the arrow keys in the Keyboard Shortcuts dialog because they do not work in all cases. The WASD keys will be cued in all three places.

Closing this issue for now.