Closed zepumph closed 3 years ago
https://github.com/phetsims/scenery/issues/945 is working well enough that I think it is a viable option. I am going to request that we discuss in greater detail, but here is a summary prior to meeting.
Proposed change works by catching the single click
when it isn't accompanied by keydown
and keyup
events. In this case, we dispatch a fake keydown
event then a fake keyup
event to the target of the click
event. In all other cases, we just handle the usual keydown
and keyup
events as they reach the browser.
This way all buttons can respond to keyup
and keydown
events. PressListener will be changed to respond to keydown
and keyup
rather than click
, and since button models use PressListener, options of PushButtonModel will work without any additional effort. And enter and spacebar will behave the same way (unless we don't want that, but the point is we have control now).
This doesn't change the fact that the behavior will still be different when a screen reader is enabled. While pressing and holding enter, we will catch lots of click events and dispatch many keydown
keyup
events to the target, so the button gets pressed many times. We can do something to prevent buttons from being fired too rapidly in this case. But the API for listening to alternative input is the same for all buttons and is more similar to pointer events.
In special cases (like usages of MomentaryButtonModel exemplified in EyeDropperNode of ph-scale), the only change will be to use something other than role="button"
so that we get accurate keydown
and keyup
events when a screen reader is in use. No other changes related to input handling would be necessary.
This approach sounds promising, thanks for all the work investigating!
This doesn't change the fact that the behavior will still be different when a screen reader is enabled.
That's unfortunate, because it's going to increase implementation and testing cost. If buttons fire only once without a screen reader, but fire more than once with a screen reader, then both behaviors need to be accounted for and tested. It's not clear how we'll account for multiple firings, but we shouldn't just let it happen and keep our fingers crossed. We need to ensure that the sim doesn't have performance problems or (worse) crash.
Before proceeding, I think that the entire dev team should be brought up to speed on this issue, so that we all know what the limitations are, what the proposal is, and how this affects use of PhET buttons in sims.
Participants of this issue set up a time to discuss https://github.com/phetsims/scenery/issues/939#issuecomment-466537529 in more detail on March 4.
Also adding dev meeting label to review with the dev team. The challenge that we are facing is that screen readers change which events are sent to the browser. Please see this comment: https://github.com/phetsims/scenery/issues/939#issuecomment-458636828 (EDIT from MK: two or three comments after that one are helpful in understanding the scope of the problem too)
Some screen readers only sends a single click
event for buttons without keydown
and keyup
events. Responding to single click
events alone doesn't let us implement the options of PushButtonModel
(fireOnDown
, fireOnHold
, fireOnHoldDelay
, fireOnHoldInterval
) for alternative input. Our recommended solution with pros and cons is listed in https://github.com/phetsims/scenery/issues/939#issuecomment-466537529.
For dev meeting we are hoping to review that proposal and how it affects use of PhET buttons in sims.
Mapping click to down/up sounds reasonable to me. It seems we have the problem of repeat clicks whether or not we map click to down/up.
This doesn't change the fact that the behavior will still be different when a screen reader is enabled. While pressing and holding enter, we will catch lots of click events and dispatch many
keydown
keyup
events to the target, so the button gets pressed many times
Can you please provide steps to reproduce this behavior? Can it be demonstrated with VoiceOver?
UPDATE: I noticed the example in https://github.com/phetsims/scenery/issues/939#issuecomment-458585395
W3C describes support for Enter and Space for buttons, see https://www.w3.org/WAI/GL/wiki/Making_actions_keyboard_accessible_by_using_keyboard_event_handlers_with_WAI-ARIA_controls#Example_1:_Using_Space_to_activate_a_button:
Adding the ARIA role "button" makes it clear that this is a button control and not a link. Because we are using an anchor element, the control is focusable and the browser will automatically call the onclick handler for the Enter keystroke. Since users expect to be able to activate buttons with either Enter or Space, a keyboard handler provides the support for activating the button via the Space character.
Note that supporting SPACE in addition to ENTER is not required by WCAG 2.0; this control would still be keyboard accessible without the key event handler for SPACE. However, supporting platform keyboard conventions for controls is strongly encouraged.
This supports PhET's decision to handle both Space and Enter, and is a more "official" reference than the Stackoverflow thread referenced in https://github.com/phetsims/scenery/issues/939#issuecomment-458652876.
We will talk about this more on Monday. I thought it was still worth having a "pre-discussion" to explain the lower level details of the issue. The dev team briefly discussed this (only 15 minutes).
Most of the discussion was explaining the how the events work for space/enter and toggling while AT is on/off. We also explained @jessegreenberg's work on the click event "catching" branch.
AP: in addition to the click catching, perhaps some buttons could have a timing like "you can't fire this faster than x." "Remove Wall" button and "Reset All" button could be potential options.
CM: maybe we can do more natively, have we tried type="submit"
?
JB: seems like there is significant value in keeping a button role right?
MK: yes, but it seems like there will be places, like momentary buttons, where we will most likely not be able to support role button.
In comment https://github.com/phetsims/scenery/issues/939#issuecomment-463725218 @pixelzoom said:
All PhET buttons currently have the options that we identified. So my feeling is that all PhET buttons need those options.
I think this is where we are having trouble communicating. My feeling is that not all PhET buttons need all the options.
From an accessibility point of view the Reset All button and the fire-on-hold button (e.g., the eye-dropper button) are different types of buttons. It would seem logical to me that they would not share the same base class. The fire-on-hold interaction is not a native button interaction. It is actually a custom interaction, one we have not yet tackled for accessibility.
Our best guidance is in ARIA authoring practices. https://www.w3.org/TR/wai-aria-practices-1.1/
Buttons are described here: https://www.w3.org/TR/wai-aria-practices-1.1/#button
Native button functionality we already use in sims:
It would great in our meeting on Monday if we can get some clarity not only for a solution that works for now, but a possible ideal solution.
@terracoda said:
In comment #939 (comment) @pixelzoom said:
All PhET buttons currently have the options that we identified. So my feeling is that all PhET buttons need those options.
I think this is where we are having trouble communicating. My feeling is that not all PhET buttons need all the options.
Agreed and agreed.
I should have said "All PhET button base classes currently support the features needed by PhET buttons." Base classes contain code that is needed to support all buttons. Individual button types use those features as needed, by overriding default values. Examples: Most buttons use the default of fireOnDown: false
, but momentary buttons use fireOnDown: true
. Most buttons use the default of fireOnHold: false
, but ArrowButton uses fireOnHold: true
. Implementing these features individually for button subclasses would result in lots of code duplication, maintenance issues, and (possibility) idiosyncratic differences across features that should have identical behavior. For the same reasons, ideally most of the a11y instrument should be (and currently is) in base classes, with subclasses overriding the default a11y where necessary.
Is that clearer?
Our communication might be improved if I provide an example of how (and why) PhET buttons are implemented. I'll use ResetAllButton
as an example, since we're all familiar with it. I'll start with the most general classes, and work my way "up" the stack to ResetAllButton
. There are 7 classes described, and I'll provide links to all of the files mentioned.
The most general classes live in the sun
and scenery
repositories. Classes in these repositories are intended to be usable by 3rd parties (outside of PhET).
As with most things in PhET, buttons use the MVC (Model View Controller) design pattern to separate model and view parts of the implementation. The model determines behavior and the view determines "look". (We can safely skip the Controller for the purposes of this discussion.) The inheritance relationships are:
model: PushButtonModel extends ButtonModel
ButtonModel - describes the most basic behavior of buttons when users interact with them PushButtonModel - adds behavior to ButtonModel that is specific to push buttons
view: RoundButtonView extends Node
Node - base class for anything that is rendered RoundButtonView - visual representation of any button that is round
RoundPushButton
puts the model and view together, using composition for the model and inheritance for the view. RoundPushButton
has an instance of PushButtonModel
(composition) and RoundPushButton extends RoundButtonView
(inheritance). So to summarize:
RoundPushButton - creates a functional push button by combining the appropriate model (PushButtonModel) and view (RoundButtonView)
So all of the above gives us something that we can use in any situation where we need a "round push button". It provides the general look of a round push button, and the general behavior for responding to user interaction. It doesn't determine specifically what happens when the user interacts; that's up to clients and subclasses to specify.
Now let's look at the button classes that are specific to PhET. They live in the scenery-phet
repository, which is where PhET-specific UI components live.
The buttons here are specialized via inheritance. ResetButton extends RoundPushButton
, and its sole responsibility is to add the icon that appears on a general "reset" button. RestAllButton extends ResetButton
, and its responsibility is to modify the look (make the button orange) and perform tasks that are general to a "reset all button". It does not know anything about fireOnDown
, fireOnHold
, the button's shape (round), or any of the other general features -- that is all handled in base classes, and ResetAllButton
would only be concerned with a base class feature if it needed to override default behavior. The client who creates a ResetAllButton
is responsible for specifying what specifically happens when the button is pressed. To summarize:
RestButton - a round push button with "reset" icon ResetAllButton - a reset button with standard color for "reset all"
The above undoubtedly seems complicated. And it is very different than buttons in web application toolkits. Back when PhET started HTML5 development, we evaluated web UI toolkits (e.g bootstrap), and determined that they were insufficient to meet the requirements of PhET sims. PhET sims require UI components that are more sophisticated and customizable. So PhET invested in writing its own button library (sun/buttons/), which is closer in style and implementation to a toolkit for standalone applications (vs a toolkit for web applications).
ARIA is a W3C (web) standard that's targeted at typical web applications. PhET sims run in a web browser, but are not typical web applications. So I think that's why we're having difficulty applying ARIA practices to PhET buttons, communicating about this issue, etc.
Hope that helps some.
Thanks @pixelzoom. Glad we agree that accessibility needs to be built into the base classes. Looking forward to Monday's discussion.
In preparation for 3/3/19 conference call, I did a deeper dive into the ARIA references that @terracoda provide in https://github.com/phetsims/scenery/issues/939#issuecomment-468624994, as well as the HTML5 specifications. I was looking for info relevant to this issue, especially ways that we might customize the ARIA button
role to better align with PhET button behavior. Here are some chunks of information and questions that I thought might be relevant. (Apologies to readers who already know this, or have already investigated these questions.)
So that's we're all on board with terminology... In the ARIA specification, “button activation” is what PhET has referred to as “button firing”. When a button is activated (fired), the action(s) associated with the button should be performed.
Regarding UX, we haven't been able to answer these 2 questions: Why does activation of Enter vs Space behave so differently? What activation behavior (if any) do users expect? Unfortunately when Enter and Space are mentioned for ARIA roles, the specification is vague and doesn't describe default behavior. For example, from the description of button
role at https://www.w3.org/TR/wai-aria-practices-1.1/#button:
When the button has focus:
- Space: Activates the button.
- Enter: Activates the button.
While I could find no ARIA specification of the activation behavior for Enter and Space keys, I did find a section in the HTML5 spec that defines Activation in general, see https://www.w3.org/TR/html5/editing.html#activation. Question: Can we customize button activation behavior?
HTML buttons have an optional type
attribute, see
https://www.w3.org/TR/html5/sec-forms.html#the-button-element. The type
attribute “controls the behavior of the button when it is activated.”
Question: Have we investigated how button activation changes depending on the value of the type
attribute?
ARIA button
has an optional aria-pressed
attribute, used to determine whether a button is pressed. See https://www.w3.org/TR/wai-aria-1.1/#button. This attribute is typically used only for toggle buttons, but perhaps we could leverage it for other buttons.
Question: Can we use the aria-pressed
attribute to provide more context for interpreting click
events? (EDIT: After further reading, using aria-pressed
for non-toggle buttons doesn't sound like a good idea.)
More about the click
event that has been mentioned extensively in this issue, followed by a couple of questions...
The click
event is a synthetic (DOM) event, specified in https://www.w3.org/TR/uievents/#click. It is synthesized in response to other events (e.g. keydown
, keyup
).
https://www.w3.org/TR/html5/webappapis.html#fire-a-click-event defines what a click
event is:
Firing a click event means firing a synthetic mouse event named click, which bubbles and is cancelable.
https://www.w3.org/TR/html5/webappapis.html#firing-a-synthetic-mouse-event-named-e defines what “firing a synthetic mouse event” means.
https://www.w3.org/TR/html5/editing.html#dom-htmlelement-click describes what happens when a click event is synthesized:
The click() method must run the following steps: 1 If the element is a form control that is disabled, abort these steps. 2 Run synthetic click activation steps on the element.
The steps involved in “run synthetic click activation steps” are defined in the Activation section of the spec, mentioned above and at https://www.w3.org/TR/html5/editing.html#activation. These steps appear to be customizable.
Question: Do any of the click
event fields provide us with useful information?
Question: Can synthesis of the click
event be customized?
ARIA examples for role=button
are at https://www.w3.org/TR/2016/WD-wai-aria-practices-1.1-20160317/examples/button/button.html#. (EDIT: Newer version of this webpage might be https://www.w3.org/TR/wai-aria-practices/examples/button/button.html)
The behavior of the "Mute" button here is interesting. If you tab to the Mute button, then press and hold the Space key, there is a brief delay, then the button activates (fires) repeatedly. This does not match the behavior that we've seen for the Space key - we've seen activation on keydown, with seemingly no way to accomplish fireOnHold.
The links to source code are also interesting. In button.js, they add event listeners for click
and keydown
to each button. Why?
Question: Why is the activation behavior different in this example? Is there something we can apply? (EDIT: See also the commits for example fixes in https://github.com/w3c/aria-practices/issues/610.)
Here's a bug report related to the difference between Enter and Space. It indicates that the button examples in https://github.com/phetsims/scenery/issues/939#issuecomment-469057113 are buggy, and the ARIA design pattern is incompletely specified.
The bug report is https://github.com/w3c/aria-practices/issues/610:
The synthetic click event that happens on native
I found this bug report by clicking the "Related Issues" link for the ARIA role=button examples at https://www.w3.org/TR/wai-aria-practices/examples/button/button.html. Note that this issue is in repository w3c/aria-practices (ARIA Practices). It was verified to be a bug, it was fixed, and the issue was closed on 10/12/2018.
Edit: Note that this bug report is the first place where I've found evidence that the Enter and Space keys are expected to behave differently. _Question: Should we contact someone from W3C ARIA to find out why Enter and Space are different, if it's a standard that shouldn't be changed, etc.?_
Great investigation and questions @pixelzoom. And glad you found the latest version of the button examples. The first link was to a "WD" or "working draft". In the latest version, on the Mute toggle button, the Space key fires/is activated on release and the Enter key fires/activates repeatedly which seems more typical, though still unexplained by the standards documentation.
@jessegreenberg and I investigated many types of button in our early investigations for BASE's custom "drag & gab" interaction, but for other reasons. We know a lot more now - both about users and about the HTML and ARIA specifications.
Exploring how activation can be customized, understanding the fullest potential of the click event, and exploring further possibilities available through the aria-pressed
attribute are all very worthy discussion topics.
Looking forward to our discussion.
@jessegreenberg or @zepumph can you please summarize the discussion from Monday, and point out the next steps for this issue? Does it still need to be discussed at dev meeting?
During discussion on 3/4/19 we reviewed why we are encountering this issue and potential solutions. We discussed how scenery/simulations would behave if we proceed with changes described in https://github.com/phetsims/scenery/issues/945. No major concerns with this came up during the meeting.
An alternative proposed during the meeting was to ignore "press and hold" and "fire on down" interaction for alternative input and just fire buttons on click
events as they hit the browser natively (which is what we are doing now). @jessegreenberg thinks it is still worth exploring https://github.com/phetsims/scenery/issues/945 since this will provide us with "press and hold" and "fire on down" buttons for alternative input, while still responding to single click
events when they are received from an assistive device. During the meeting it was mentioned that the downside of this is that behavior will change when an AT is in use and therefore increase the cost of development and testing.
Here are the next steps: After the meeting @zepumph mentioned that he encountered different results than I had reported in https://github.com/phetsims/scenery/issues/939#issuecomment-458585395, we should understand why that is the case.
@pixelzoom identified a number of resources and questions in https://github.com/phetsims/scenery/issues/939#issuecomment-469051241 https://github.com/phetsims/scenery/issues/939#issuecomment-469053432 and https://github.com/phetsims/scenery/issues/939#issuecomment-469058519, those should be reviewed before more work is done.
After doing those things we can decide whether or not to proceed with #945.
@jessegreenberg, I also shared with you, @pixelzoom and @zepumph comments from the WAI ARIA interest group which included a mark-up idea that we have not yet explored, nesting a button
element within a parent element with role=application
.
<div role="application">
<button>Squeeze Eye-dropper</button>
</div>
This mark-up could potentially be useful for the "press and hold" and "fire on down" buttons once we get them for alternative input.
Just adding it, since it is related to the issue.
Thanks for collecting resources and questions @pixelzoom. Some of this we have already considered but all of it is worth looking into.
I am reviewing the questions and links listed in https://github.com/phetsims/scenery/issues/939#issuecomment-469051241, https://github.com/phetsims/scenery/issues/939#issuecomment-469053432, https://github.com/phetsims/scenery/issues/939#issuecomment-469057113 and https://github.com/phetsims/scenery/issues/939#issuecomment-469058519
Question: Can we customize button activation behavior?
Not to my knowledge. https://www.w3.org/TR/html5/editing.html#activation spec is for the user agent (browser), I don't see anything there usable by web developers.
Question: Have we investigated how button activation changes depending on the value of the type attribute?
The type
attribute controls the behavior of the button with respect to a containing <form>
element. It doesn't have an impact on the DOM events. I created https://jsfiddle.net/ne2xrhyp/1/ to verify, this example has submit
, reset
, and button
buttons. Event behavior is the same for all.
Question: Can we use the aria-pressed attribute to provide more context for interpreting click events? (EDIT: After further reading, using aria-pressed for non-toggle buttons doesn't sound like a good idea.)
+1 to the edit. But I also verified that aria-pressed
has no impact on events when an AT is in use.
The steps involved in “run synthetic click activation steps” are defined in the Activation section of the spec, mentioned above and at https://www.w3.org/TR/html5/editing.html#activation. These steps appear to be customizable. Question: Do any of the click event fields provide us with useful information? Question: Can synthesis of the click event be customized?
These are steps for the user agent. The fields available for web developers are listed in MouseEvent(because click
uses the MouseEvent
interface even though it is commonly triggered with keyboards and assistive tech).
ARIA examples for role=button are at... In button.js, they add event listeners for click and keydown to each button. Why? Question: Why is the activation behavior different in this example? Is there something we can apply?
It is mentioned in https://www.w3.org/TR/2016/WD-wai-aria-practices-1.1-20160317/examples/button/button.html# that keydown
is added to handle keyboard support and click
is added to handle mouse clicks and screen readers. @terracoda and I tried something similar in https://github.com/phetsims/scenery/issues/939#issuecomment-461824374. It could be a way to get enter and spacebar to behave the same, but it has the same problem with AT in that it won't support the options of PushButtonModel.
Question: Should we contact someone from W3C ARIA to find out why Enter and Space are different, if it's a standard that shouldn't be changed, etc.?
Yes, we could do that. We happen to know the author of that issue who is a member of the ARIA WG. Differences between enter and spacebar go beyond buttons and the web. For instance, checkboxes only activate with spacebar, they don't respond to enter. I just verified that was true in a native Windows app.
Actually, I had forgotten that @terracoda already reached out to the ARIA group about this. https://lists.w3.org/Archives/Public/w3c-wai-ig/2019JanMar/0086.html
No matter whata we do, somewhere in button code we will need to change the markup for the button so that it looks like
<div role="application">
<button>BUTTON</button>
</div>
If we don't go with #945, somewhere in button code we will also have to opt out of click
listeners. We could have an option in PressListener like activateWithKeyboardEvent
that disables the click
listeners and uses keyup
and keydown
listeners instead.
If we do this, it will make button code a little more complicated, keyboard use without a screen reader will generally not support PushButtonModel options. But we avoid doing something tricky and PhET specific for a11y in scenery. And we avoid adding differences in sim behavior when a screen reader is enabled vs disabled.
I think we should just do that. This will also set us up for switching to #945 strategy if we need to later, but avoids trickiness for now.
@zepumph can we talk at tomorrows dev meeting to finalize?
Notes from 4/8/19 discussion:
During discussion on 4/8/19, @samreid suggested that rather than having some of our buttons use use unique markup and input handling that we modify the user interface (both visual and PDOM) to support different modes of interaction. For instance, for the StepButton we could also have a "slow motion" button that changes the behavior of step to work better with single click activations. However, it is unclear how this would work because the step button will still respond to click events at the interval that the assistive device sends events to the browser. And modifying the UI will be more time intensive.
By keeping our current system, we get the semantic button interaction that has been tested with users and we know works well. We don't have to change our input system for a11y or doing anything that is PhET specific. For specific buttons, we can support PushButtonModel options doing the following:
Some buttons will have different markup that looks like
<div role="application">
<button>BUTTON</button>
</div>
And PressListener will be modified to optionally support keydown
and keyup
events rather than click
events.
If in user interviews we determine that this is too confusing for users, we will be in an OK position to change approaches again at that point.
@zepumph recommended that these decisions should be documented in style guide document files (like PushButton.md) for review from relevant people in the future.
@samreid also provided these notes from discussion:
We additionally discussed these examples:
- Changing the pDOM elements for
MomentaryPushButtonModel
buttons to toggle buttons, so on the first click, the button is pressed and on the second click, the button is released.- Breaking out the “Step” button with 2 mode of interaction (press once to step once vs press and hold to go in slow motion), to two buttons each with one mode of interaction (one button that steps once when pressed once and another button or radio button for slow motion).
We concluded that
<div role="application">
can be used to get things working “out of the box” but that should be “opt in” and for components that use that strategy, we should consider other alternate patterns during the a11y design process, because “press and hold” is (a) not necessarily something all users can do and (b) an unconventional accessibility pattern compared to standard html components.
We concluded that
can be used to get things working “out of the box” but that should be “opt in” and for components that use that strategy, we should consider other alternate patterns during the a11y design process, because “press and hold” is (a) not necessarily something all users can do and (b) an unconventional accessibility pattern compared to standard html components.I agree that
<div role="application">
should be an opt-in option.<div role="application">
is meant for custom interactions. In interviews thus far (BASE, Friction, Faraday's Law) this role has not yet been understandable or easily operable by users. Explicit guiding instructions are always needed and these instructions vary by platform (e.g., keyboard, touch).I also agree with (a) and (b) in the above comment.
We need to tread very carefully with
<div role="application">
.
Some buttons will have different markup that uses role="application" And PressListener will be modified to optionally support keydown and keyup events rather than click events.
This means that we are proceeding with our current input system for catching click
events and we will add support for keydown
keyup
in PressListener when we require them. This issue no longer needs to block publication.
The work described in https://github.com/phetsims/scenery/issues/939#issuecomment-483781361 has been moved to #1117. Closing this issue.
Reopening based on the TODO in the code:
* TODO: This may change after https://github.com/phetsims/scenery/issues/939 is done, at which point
* `click` should likely be replaced by `keydown` and `keyup` listeners.
Thanks, I updated the link to point to the new issue referenced in https://github.com/phetsims/scenery/issues/939#issuecomment-729207681
From https://github.com/phetsims/scenery/issues/934, in the browser they are different, @pixelzoom and @zepumph discussed at one point whether or not that adds value to phetsims, it would be good to discuss that further, and then implement from there. This also may relate to how we are handling sun buttons, and that should be looked at in this issue to.
Related potentially to https://github.com/phetsims/sun/issues/424, there is likely little use for the default behavior of Enter to be fire multiple events.
Even though it is browser default, we know we want to support users at PhET that are younger, and may not have as much keyboard experience, it could easily be overwhelming to have a button fire many times without a good, pedagogical reason to. Expanding on that, then why would we make that the default behavior for Enter?
I think from discussing with @pixelzoom that we are mostly in agreement (correct me if I'm wrong) that the default behavior for both should (a) be the same, and (b) be a single click for buttons. See https://github.com/phetsims/scenery/issues/931 for adding support for options that would deal with "fire on hold" features.