Can't move John's arm or leg when using VoiceOver

liammulh commented 3 years ago

Test Device

MacBook Air

Operating System

macOS 11.2.3

Browser

Safari 14.0.3

Problem Description

For https://github.com/phetsims/QA/issues/645. I can't move John's arm or leg using my normal technique, which is to navigate to the slider, enter the slider using control-option-shift-down, and then use the left and right arrow keys to move the slider. I'm able to move John's arm and leg in the published John Travoltage. See the video below.

Visuals

Google Drive Link

Troubleshooting Information

Name: ‪John Travoltage‬ URL: https://phet-dev.colorado.edu/html/john-travoltage/1.6.0-preferences.4/phet/john-travoltage_en_phet.html Version: 1.6.0-preferences.4 2021-04-28 22:49:44 UTC Features missing: touch User Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_6) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/14.0.3 Safari/605.1.15 Language: en-us Window: 1264x795 Pixel Ratio: 1/1 WebGL: WebGL 1.0 GLSL: WebGL GLSL ES 1.0 (1.0) Vendor: WebKit (WebKit WebGL) Vertex: attribs: 16 varying: 15 uniform: 1024 Texture: size: 16384 imageUnits: 16 (vertex: 16, combined: 32) Max viewport: 8192x8192 OES_texture_float: true Dependencies JSON: {}

terracoda commented 3 years ago

@liam-mulhall, just curious how you got the pink highlight on the Leg Swing before trying to rub the foot with the arrow keys.

Did you move focus to the Leg Swing with the keyboard (Tab key)? Or did you click on the Leg with the mouse with Interactive Highlights enabled?

liammulh commented 3 years ago

@terracoda, I used the tab key.

terracoda commented 3 years ago

Thanks @liam-mulhall, this doesn't happen for me when using the latest dev version (john-travoltage/1.6.0-preferences.4/) with Safari and Voice Over.

Noting my system is older MacBook Pro, MacOs 10.13.6

jessegreenberg commented 3 years ago

The video looks pretty weird @liam-mulhall, I am not sure what is happening. But I just tried with master and in https://phet-dev.colorado.edu/html/john-travoltage/1.6.0-preferences.4/phet/john-travoltage_en_phet.html and was not able to get this to happen.

With VO in Safari, If I tab to the slider and use the arrow keys the slider moves just fine. If I tab to the slider and press "ctrl" + "option" + "shift" + "down arrow", I am still able to move the arm and leg. I am using macOS 11.4.

Since neither I nor @terracoda were able to see this I am going to close and assume this was either temporary or a platform quirk. Feel free to reopen if it happens again @liam-mulhall .

jessegreenberg commented 3 years ago

@Nancy-Salpepi noticed this issue again while testing https://github.com/phetsims/qa/issues/676. VO says to press keys that don't actually move the slider.

terracoda commented 3 years ago

@liam-mulhall, can you make sure you have full keyboard nvaigation enabled in System>Preferences>Keyboard and also that you do not have VO's QuickNav enabled.

Pressing the left-right arrow keys back and forth quickly can enable QuickNav. With QuickNav on, the arrow keys move the cursor instead operating the slider.

You can easily toggle between QuickNav on and QuickNav off with the right and left arrow keys. You probably knew this, but just checking

jessegreenberg commented 3 years ago

Discussed with @emily-phet -

@emily-phet confirmed that this has been an issue for her. We will check and see if this is a problem for "vanilla" HTML sliders. If it is, it is VO bug and we will submit a bug report. If not, we will inspect our implementation.

Nancy-Salpepi commented 3 years ago

I just wanted to add, that like Liam, when I went into the published version of the sim, the VO + arrow keys worked to move the leg back and forth.

terracoda commented 3 years ago

I verified this is an issue for me, too.

With VO enabled I can use regular Browser navigation and slider operations. That is, I can Tab to the Arm and Leg Swing sliders, I can move them with the Arrow keys, I can move them in bigger steps with function Up and Down Arrow keys. Function Left and Right Arrow keys are not working due to a known bug noted elsewhere.

When I use VO commands, I can navigate to the Arm and Leg with:

VO + Cmmd + J

I can get into the slider with:

VO+Shift+Down Arrow

Once in the Slider things are weird:

VO+Arrow keys do not move the slider
Arrow keys alone move the slider
Option+Command+Arrow Keys move slider
Option+Arrow Keys move slider
Command + Arrow Keys move slider
Control+Option+Command+Right Arrow opens the Rotor tool

I do not know why VO Keys are required by VO when in the slider. Since the VO keys have worked before to move sliders, maybe we can figure out what has changed. Did our slider change or did VoiceOver and Safari change?

jessegreenberg commented 3 years ago

I just tested with macOS Safari and had exactly the same results as @terracoda. I just tried using a vanilla HTML slider and noticed something different about using "VO + arrow keys" to move a slider.

For this html:

<input type="range" id="slider">
<label for="slider">My Slider</label>

Press VO + Shift + Down Arrow to go "In Slider"
Press arrow keys to increment/decrement the value by 1.
Press page up/page down to increment/decrement the value by 10.
Press VO + arrow keys to increment/decrement the value by 5!!

So the behavior for macOS is to change the step size when using VO + Arrow keys. So I am guessing VO is trying to change the arm/leg position when using "VO + arrow keys", but it isn't a step size we counted on and so we are rounding back to the previous appendage value/angle. Ill do some testing now to see if that is the case.

jessegreenberg commented 3 years ago

I think I found a fix for this in the above commit, moving sliders with VoiceOver keys is working well now. I tested every browser we support after this change, and then Safari + VoiceOver and it is working well. Adding label to verify by QA.

Nancy-Salpepi commented 3 years ago

I have no issues moving the leg and arm using the arrows once in the slider. As long as I'm moving the arrow keys it doesn't matter what other buttons I may be pressing (VO buttons + arrow, option + arrow, command + arrow, any letter + arrow). I don't see that as an issue, but @jessegreenberg I'll let you decide.

jessegreenberg commented 3 years ago

Thanks @Nancy-Salpepi - What you described sounds correct to me and matches what @terracoda found in https://github.com/phetsims/john-travoltage/issues/414#issuecomment-889962888. Before changes for this issue pressing VO keys did not move the arm/leg at all and that is what we were trying to fix. I think this can be closed.

phetsims / john-travoltage