search

w3c / aria

Accessible Rich Internet Applications (WAI-ARIA)
https://w3c.github.io/aria/
Other
637 stars 123 forks source link

Need an attribute for direct interaction elements on a touch screen #1215

Open minorninth opened 4 years ago

minorninth commented 4 years ago

Screen readers with touch screen support typically include a "touch exploration" mode where you can tap or slowly drag around the screen and listen to feedback on what you're touching, before it activates. To actually activate, you double-tap.

There are a few cases where this is undesirable - like a virtual keyboard, or a signature pad. In those cases you want gestures to be passed through directly.

Some native accessibility APIs already have a way to specify this, like UIAccessibilityTraitAllowsDirectInteraction on iOS.

We should have a similar ARIA role or attribute for this.

I think at one point I suggested role="key" for a keyboard key, but I think the concept is more generic. Besides the signature pad, it could also be useful for a musical instrument, a game, or many other things.

It seems related to aria-interactive. The idea behind aria-interactive is that the control has its own keyboard support on a desktop computer. This idea is that the control has its own touch event support on a mobile touch device. They're quite similar!

So one idea would be: aria-interactive="touch keyboard mouse", etc. where you choose from various tokens.

Or, we could make it separate, like:

role="touch" role="directtouch" aria-touch="announce" vs aria-touch="activate" aria-touchactivate="true"

frastlin commented 6 months ago

There are no apps that only require one finger to use with Voice Over. A basic news app has 4-finger single tap to get to the top of the screen, swipe down with 2 fingers to read content, single tap with 2 fingers to stop speech, and you use the rotor to adjust your reading level which is rotate with 2 fingers. Obviously, if you're asking Voice Over to pass through its gestures, it is on you to provide the needed functionality for the Voice Over user to use your app. This feature should only be used for non-semantic interfaces, and Voice Over users should always have a way to exit out of direct touch. I think an all-or nothing experience is very much desired for games, like your card app. Of course you're going to need to have a guide like the other 829 Audio Games do to help people learn your gestures, but that's normal and expected. If you want to have that functionality for your game where you swipe with one finger, then that's great. Our map has over 60 key commands, and power users are going to want access to those commands through custom gestures on the interface. Someone can slowly use our interface with one finger, but it's slow and people who use it frequently are going to want to have custom gestures. I even have blind power users who hate the fact that WCAG requires focus to be at the top of the page when you open dialogues. They want minimal keystrokes to do the most things as fast as possible. Just as long as Voice Over users can exit direct touch mode at any time, there should be no problem with passing through almost all gestures to an app. This is what happens if you focus on a button in the browser now, all keystrokes get passed to that button. This is perfectly fine and our team of blind developers and testers want a way to do the same for touch gestures.

zphrs commented 6 months ago

Thank for for explaining the demand, the convenience, and the normality of supporting multi-touch inputs. Now I totally get where you're coming from and I support having multi-touch as part of the spec.

I guess my question then is what default gesture do you think could be a good alternative for "tab" to move focus away from the element for touch devices? Naturally tapping with one finger elsewhere on the display would work well for touch areas which do not take up the whole display and I assume that the typical tab button will suffice on keyboard devices to tab away from the direct input unless of course that tab key is listened to and preventdefault() is called on that keypress event - in which case the developer would need to call blur() on the directinput element to manually reimplement that tab away behavior.

You mentioned that in your app you use a swipe from the bottom combined with a rotor gesture. I guess another question I have for you is whether that secondary rotor gesture is also really needed to release input from a usability standpoint or whether just a swipe up could suffice to break the user out of that touch area.

I recognize that ultimately the implementation will come down to how android and iOS handle it but I think a possible good option is that whatever the typical "go home" gesture on the touch device is (in addition to the tab input for touch devices with a keyboard) will deactivate the direct input and resume treating inputs with the typical browser behavior. I also assume that developers could add additional triggers to re-enable typical pointer input by calling focus() on a different element or calling blur() on the directinput element. I also assume that similar to the keyboard, developers could call preventdefault() on the touch/pointer event if they want to prevent the default deactivation from swiping up from the typical "go home" gesture.

What are your thoughts on this proposal @frastlin, and is there a better option that you can think of for exiting the directinput mode?

frastlin commented 6 months ago

Entering and exiting direct touch should really be a screen reader's responsibility, but yes, I agree that the go home command would be a good exit direct touch mode gesture. Direct touch currently allows this already, but it will minimize the application. To enter direct touch, the user would just activate it from the rotor like normal. Direct touch needs to be the entire screen, it can't be just one area, which is why it's important to have the easy way of entering and exiting direct touch. This is because there's no way for the blind user to know where the direct touch area is, especially since they're probably swiping right to the element instead of finding it on the screen. All Audio Games use the entire screen to handle gestures, so again, it's expected that direct touch will put all gestures into the target element, no matter where the gestures performed on the screen.