Need an attr to indicate element activation triggers audio, video, etc.

[jnurthen]

https://www.w3.org/WAI/ARIA/track/issues/603

We need a way to indicate that element activation (e.g. pressing a button) initiates or triggers some media like audio or video. This will allow screen readers, for example, to know to suppress speech confirmation of a press, so that they don't speak over the beginning of a video, music file, captcha, timer, or other media resource.

Either binary: aria-startsmedia="[true | false]"

Or more specific: aria-initiates="[audio | video | none]"

I'm not particular to any specific attribute name, but "triggers", "startsmedia", and "initiates" may be appropriate.

Note: On iOS, the binary version would be equivalent to UIAccessibilityTraitPlaysSound. https://developer.apple.com/library/ios/documentation/UIKit/Reference/UIAccessibility_Protocol/Introduction/Introduction.html#//apple_ref/doc/c_ref/UIAccessibilityTraitPlaysSound

[cookiecrook]

We’ve received more requests for this from web developers. Maybe 1.3?

The “startsMedia” trait might provide a better name for the ARIA attr than the “playsSound” trait.

https://developer.apple.com/documentation/uikit/uiaccessibility/uiaccessibilitytraits/1620204-playssound

https://developer.apple.com/documentation/uikit/uiaccessibility/uiaccessibilitytraits/1620173-startsmediasession

[cookiecrook]

Will propose this if no further comments. aria-startsmedia="[none | video | audio | call | earcon | …]"

audio/video/call values would be equivalent to the "starts media session" API. Pressing the button with AT doesn't speak a confirmation, because a media session is starting.

Bikeshed earcon (or icon, as in sound icon) would be equivalent to the "plays sound" API… so that AT need not play its own sound icon when pressed (usually a click), but should still perform the other expected behaviors (some screen readers, for example, speak a confirmation, like "pressed [label]")

[cookiecrook]

Thinking on this more, there is no functional AT difference between video | audio | call so maybe combine those in a single session value? earcon is a kitschy name, so maybe soundicon or just icon? The v1 could be:

aria-playsaudio="[ none | soundicon | session ]"

Update Jan 4: Previously proposed audio/video/call values are equivalent to the "starts media session" API, so a single value for session seems appropriate and the simplest solution. Apple hasn't found a need to differentiate these types yet, but more token values could be added to the Web API at a later time if a need arises.

The attribute name change "plays audio" is intended for clarity. For example, there is no known reason to use this on a video that contains no audio, since an AT's sounds and speech wouldn't conflict. Please comment if you have any concerns or other proposals to consider. Happy new year!

[cookiecrook]

@jcsteh Does NVDA or JAWS have similar functionality? Trying to make sure a second AT besides VoiceOver would use this API.

[jcsteh]

NVDA doesn't and I don't think JAWS does either, but I can't be certain about the latter. That said, it could be argued that it's a chicken and egg situation: there's never been an API to facilitate it, so no one would have ever considered it.

What does VoiceOver suppress in this case? Usually, I don't hear VoiceOver speak when I press a button, just the double tap earcon. Does the trait actually suppress that earcon?

[cookiecrook]

Yes, and when I filed the original issue back in 2013, Mac VO also spoke "press [button label]" but now just plays the earcon to align with the iOS VO user interface.