Alt input is broken for momentary buttons.

[pixelzoom]

I have a RoundMomentaryButton in ph-scale. It's the button on scenery-phet DropperNode, and looks like this:

RoundMomentaryButton and RectangularMomentaryButton) do not currently work correctly with alt input. A momentary button is a toggle button that is “on” on pointer down, then “off” on pointer up. The equivalent for keyboard seems like it would be “on” when the spacebar is pressed, “off” when the spacebar is released, but I don’t know if that’s possible.

This is a prerequisite for ph-scale, which is included in the PhET-iO batch release. And I believe that ph-scale is towards the top of the deploy list.

Assigning to @jessegreenberg and @zepumph. Feel free to reassign to whoever is responsible for alt input of sun buttons, or consult with @kathy-phet.

[pixelzoom]

Sky#a11y-dev discussion:

Michael Kauzmann I don't believe that we have gotten to momentary buttons yet, checking

Michael Kauzmann I see that on space, it is the keyup that triggers the button, but with enter, it is key down, so by holding enter, you get momentary behavior. Is that what you see?

Chris Malley Yeah, I’m digging into it. Looks like there were some assumptions made about buttons that didn’t consider this type of button. Adding this to RoundMomentaryButton’s default options:

      listenerOptions: {
        press: () => console.log( 'press' ),
        release: () => console.log( 'release' )
      },

Both callbacks fire when I release the spacebar. (edited)

Michael Kauzmann But it does work with enter.

Chris Malley Checking… And is that how you want alt input to work for momentary buttons, or just a happy coincidence? Doesn’t seem very intuitive.

Chris Malley Using Enter, I see the button toggling multiple times, seeming randomly. Certainly not usable.

Chris Malley To clarify… I’m pressing and holding just the Enter key, then releasing it when I want the button to toggle off.

Chris Malley And the button does change state, but its listeners do not fire. In the case of the dropper, no solution comes out of the dropper while the button is in the “on” state.

Michael Kauzmann I see. Ok good to know. I didn't realize that it didn't let a small amount out each time the enter button fired.

Chris Malley Can you explain why a PressListener press callback fires when the spacebar is released? Is that expected? It seems like listenerOptions (above) is the way this should be addressed.

Michael Kauzmann We are at the mercy of how browsers deliver their "click" event. This is given on up for space, and down for enter. If we wanted to heavily optimize, you'd THINK we could listen to keyup and keydown, but when using screen readers, we don't have that information, just the click event.

Chris Malley I see. Thanks for clarifying.

Michael Kauzmann So the bug for that issue is to make sure that just a bit can come out based on a single-point callback, instead of relying on the start/stop events being at least >0ms apart. (I think)

Michael Kauzmann But I also just have a feeling that there is a reason why we haven't gotten things working just yet. My guess is that it may open a can o insert favorite gross thing.

Chris Malley I’m not clear on what you mean by “just a bit coming out”. The user should be able to “hold the button down” (however that is done via the keyboard) for as long as they want.

Michael Kauzmann Right, but in terms of events, for mouse it is "on" and then "off" but for keyboard it is just "click", and then if you hold it down it is "click, click, click, click, click, click, click, click, click". So we need to discretize that so that the button responds in a small way as being "on" for a bit of time each time there is a click event.

Chris Malley It also seems like a usability problem to use spacebar for most buttons, and Enter for momentary buttons. As well as having implications for the “Basic Actions” section of keyboard help, where this use of Enter is currently not mentioned. (edited)

Chris Malley Perhaps we should consider making momentary toggle buttons behave like standard toggle buttons, for alt input. Spacebar on, spacebar off.

Amy Rouinfar I like the idea of having momentary buttons behave like standard toggle buttons for alt input.

Michael Kauzmann Perhaps with an option like "timePerAltInputPress" with a good default

[terracoda]

I am so glad we are looking at the dropper button interaction, aka "momentary button".

The PhET code base has many more button interactions than want can be represented natively in HTML, so we might need to think outside the box a little.

In a browser, both Enter and Space are keys that activate buttons. And as you noticed they do not function in exactly the same way. In contrast, only Enter can activate a link and only Space toggles a checkbox. These associations with actual HTML control elements are well-known to people who rely on the keyboard.

If the control is communicated visually as a button (and in the code as a button), both the Enter and Space keys will need to work with the button. Keyboard users will likely try either or both.

I think we need to think about what the function of the eye-dropper is meant to allow the learner to accomplish, e.g. constant stream of drops, and/or a quick single drop.

I think we can make the Enter key work differently from the Space key. I think we did that with the Step button in the Timing Control Node.

Let's ask what does the eye-dropper button need to do and then see if that function can be mapped to the native key presses that work for an HTML button interaction.

[terracoda]

And by HTML button, I mean, any HTML code that is communicated as a button, for example (and this is rough):

<button>Eye-dropper</button>, communicated as "Eye-dropper, button"
<button aria-pressed="false">Eye-dropper</button>, communicated as "Eye-dropper, toggle button"
<button aria-pressed="true">Eye-dropper</button>, communicated as "Eye-dropper, selected toggle button"
<button aria-pressed="false" role="switch">Eye-dropper</button>, communicated as "Eye-dropper, off, switch"
<button aria-pressed="true" role="switch">Eye-dropper</button>, communicated as "Eye-dropper, on, switch"

[terracoda]

We can make custom widgets that look like buttons. It's harder to make a custom widget sound good with a screen reader, so if we can find a way for a button to work well for the eye-dropper interaction, I think that will be a good thing.

[terracoda]

I am wondering if Enter can turn it on/off to squirt a little and holding Space can be used to fire a continuous stream? Would that be a good thing for users and work in a browser?

[terracoda]

Oh, that should be the other way around and more like this:

• Enter & Space might both turn the eye-streamer button on/off to squirt a little - this would require 2 toggles - once to on, and once to off.
• Pressing & holding the Enter key would allow for a continuous stream.
• Pressing & holding Space key doesn't do anything as a key-up is required to fire an interaction with Space key.

The questions I have are:

1. Would the following key presses make sense for the sim and learners:
   • the Space key behaves as per usual toggling the button on then off with and key-up event, and then
   • the Enter key behaves more like the mouse/touch interaction and starts streaming on the down event and it stops steaming on the up event.
2. Can we use a native button element to create this type functionality that will work across browsers and screen readers?

In our current custom drag interactions, my understanding is that we have currently two options:

• Use a Grab button to first grab (pick-up/ activate) the object and enter a dragging mode (an on mode). An intentional release is required to release the object (or turn it off). This technique is used for things like the Balloons in BASE. And has proven to work well with blind learners using screen readers. Extra supports are implemented for visual learners.
• Skip the grab button and go directly to a dragging mode (a on mode) using the Arrow keys and WASD keys. This type of interaction is used for the Both Hands in RaP. In that sim, the interaction is scaffolded by two sliders (Left Hand and Right Hand) that precede it. Custom widgets like these are generally not communicated to users in consistent ways when using screen reader software. That said, browser support can improve over time.

[jessegreenberg]

Can we use a native button element to create this type functionality that will work across browsers and screen readers?

I had the same question, I am worried that some AT won't give us keydown/keyup events for button elements. Profiling DOM events while using screen readers is the first step for solving this.

I remember reviewing this case with the team and we predicted it would be difficult. Press and hold is generally not done (even discouraged) for accessibility and it does not have good native support with screen readers. How would press and hold even work with iOS VoiceOver gestures?

[zepumph]

Subset of https://github.com/phetsims/scenery/issues/1298

[zepumph]

It isn't clear to me that this is in fact high priority. @pixelzoom is this not longer pressing because ph-scale is proceeding without alt-input?

[pixelzoom]

Priority is not my call. While it's no longer an issue for ph-scale (which is imo unfortunate), I don't know what sims may or may not need this to proceed. Please ask @kathy-phet.

[terracoda]

Ok, this issue is a priority again as we design description for phSB.

[terracoda]

@jessegreenberg said:

I remember reviewing this case with the team and we predicted it would be difficult. Press and hold is generally not done (even discouraged) for accessibility and it does not have good native support with screen readers. How would press and hold even work with iOS VoiceOver gestures?

Agreed, requiring a hold is not a robust inclusive design option.