WICG / aom

Accessibility Object Model
http://wicg.github.io/aom/
Other
571 stars 59 forks source link

First thoughts on AccessibleElement interface #8

Open alice opened 8 years ago

alice commented 8 years ago

(Just copying stuff over from email as I find it easier to read in this format)

alice commented 8 years ago

(From Dominic)

While we're not done with the use cases, there seems to be large consensus that one of the most important use cases is to support the ability to change accessibility attributes of DOM elements programmatically, and to create virtual accessibility elements.

Let's dive right into the technical details of what that might look like.

Alex has proposed adding a new property to the Node interface:

partial interface Node {
    readonly attribute AccessibleElement? accessibleElement;
};

Ignoring for the moment what we call it (Alex called it a11ement, but I'm using accessibleElement just to be more clear for now), I think the general idea that every DOM node should have a single new property to give you access to the node's accessible element is good. I think it's definitely preferable to the idea that every Node would have separate properties reflecting all of the ARIA attributes like role, etc.

If there are any objections or alternative ideas (aside from the name of the property or the name of the class), stop here and speak up.

Otherwise, let's dive into what AccessibleElement should contain.

For the moment I do not want to worry about all of the possible properties an AccessibleElement must have. For now let's stick with role and name since those are pretty well-defined and hopefully not too controversial. So we may have something like this:

interface AccessibleElement {
             attribute DOMString          role;
             attribute DOMString          name;
};

So if you wanted to log the role of the html body element, you could write:

console.log(document.body.accessibleElement.role);

Again, if there are major objections so far, stop here and let me know!

But if this all seems reasonable so far, let's talk about how we'd change the role of an element, because there are several ways that could be done.

How to change the role of an existing DOM element

(1) Alex's idea is that an AccessibleElement would have a "source":

interface AccessibleElement {
             attribute DOMString          role;
             attribute DOMString          name;
             attribute AccessibleSource?  source;
};

The idea is that AccessibleElement is not mutable, but AccessibleSource is an object that would let you specify accessibility attributes you want to modify. So you could do something like this:

var customButton = document.querySelector('#customButton');
var axCustomButton = element.accessibleElement;
console.log(axCustomButton.role);  // prints "div"
axCustomButton.source = {'role': 'button'};
console.log(axCustomButton.role);  // prints "button"

One cool thing about this approach is that you could define the role using a getter instead, which could then compute the role dynamically if you want:

axCustomButton.source = {get role() { return 'button' }}

Presumably you could also create a prototype and then set the accessible source for a whole bunch of elements to an instance of that:

for (var i = 0; i < listItems.length; i++)
  listItems[i].accessibleElement.source = new AccessibleListItem(i);

(2) Another idea to throw out would be to just allow accessibleElement itself to be mutable and replaceable directly. So to change the custom button's role, you'd just do this:

var customButton = document.querySelector('#customButton');
var axCustomButton = element.accessibleElement;
console.log(axCustomButton.role);  // prints "div"
axCustomButton.role = 'button';
console.log(axCustomButton.role);  // prints "button"

You could replace role with a getter using this syntax:

Object.defineProperty(axCustomButton, 'role', {
  get: function() {
    return 'button';
  }
});

Finally if you wanted to replace all of the functionality of accessibleElement you could do that:

customButton.accessibleElement = {
  get role() { return 'button' },
  get name() { return 'name' },
}

With this second proposal, it seems like it'd be harder to imagine providing an "incomplete" AccessibleElement - for example you'd break the hierarchy if you didn't implement something like parent / children. With Alex's idea, above, AccessibleElement could always implement more critical things like the tree structure, and AccessibleSource would override things you want to, while anything you choose not to override would be inferred from the underlying Node (if any) or from explicit tree structures.

I think Alex's idea (1) is a little harder to get your head around at first, but it's potentially more foolproof, it's more compatible with the idea of explicitly constructing virtual trees. My alternative idea (2) is simpler to understand but potentially requires the author to be more careful when implementing virtual accessible nodes.

Thoughts? Alternative ideas?

alice commented 8 years ago

(From Chris)

The extra level of indirection with an accessible source seems like it would make adoption more confusing.

Wouldn’t we implement default values for many properties. i.e.) your accessibleElement.role would inherit role from the DOM element but you could override that if desired. More importantly, parent and children nodes would come for free unless you overrode with

button.accessibleElement.accessibilityChildren = [ ]

where you could then create an entirely fake accessible tree

Thx

alice commented 8 years ago

(From James)

Alex has proposed adding a new property to the Node interface:

partial interface Node {
    readonly attribute AccessibleElement? accessibleElement;
};

Have you considered an accessor instead of a property? This might fit better with your "instantiate accessibility only on demand" use case, and could also fit will well with the plan for virtual subtrees. For example, standard DOM elements would get element.accessibleElement() provided by the browser, but a canvas app author could overwrite the native method:

myCanvasElement.accessibleElement = function() { return myCustomAccessibilityViewController; }

This was the main goal behind our previously mentioned "accessibility delegate" idea:

myCanvasElement.setAccessibilityDelegate(myCustomAccessibilityViewController);

For the moment I do not want to worry about all of the possible properties an AccessibleElement must have. For now let's stick with "role" and "name" since those are pretty well-defined and hopefully not too controversial. So we may have something like this:

Minor nit: name should be label.

ARIA, iOS, and more recently OS X have been using "label" to avoid any baggage and ambiguity that come with the other possible terms like name, title, and description. For example, @name in HTML is a non-unique identifier.

interface AccessibleElement {
             attribute DOMString          role;
             attribute DOMString          name;
};

So if you wanted to log the role of the html body element, you could write:

console.log(document.body.accessibleElement.role);

Again, if there are major objections so far, stop here and let me know!

No objection, but I'm curious what this should return in cases where there is not a 1:1 match with an ARIA role. For example, some custom web component, or the native <video> element? Should we consider the role property is limited to those defined by ARIA? I think so, but it leaves us with a lot of gaps for WebDriver. We could consider exposing the user agent's internal role as another property or accessor.

let axEl = document.createElement("video").accessibilityElement();
axEl.role // ""??? The WebKit Inspector calls this "No exact ARIA role match." It's not unknown; it just doesn't match a standard.
axEl.engineRole // "videoRole" in WebKit/Blink, but not intended for standardization, so Gecko/Edge could return anything here.

Finally if you wanted to replace all of the functionality of accessibleElement you could do that:

customButton.accessibleElement = {
  get role() { return 'button' },
  get name() { return 'name' },
}

This is similar to the accessor method.

customButton.accessibleElement = function(){
    return {
        get role() { return 'button' },
        get name() { return 'name' },
    };
};

One benefit to the method over the property is that developers won't inadvertently trigger execution of accessibility code just by inspecting the properties of the containing Element. (Is there another way to keep the property variant from incurring the performance hit when the parent element is inspected?)

With this second proposal, it seems like it'd be harder to imagine providing an "incomplete" AccessibleElement - for example you'd break the hierarchy if you didn't implement something like parent / children. With Alex's idea, above, AccessibleElement could always implement more critical things like the tree structure, and AccessibleSource would override things you want to, while anything you choose not to override would be inferred from the underlying Node (if any) or from explicit tree structures.

I think Alex's idea (1) is a little harder to get your head around at first,

This is my biggest concern for Option #1. Is there any precedent for this pattern in existing web APIs? One of the reasons we never publicly mentioned the "accessibility delegate" idea was that it didn't feel very "web-like." and web authors wouldn't understand it.

In order for a new web API to be successful, it needs to be feel familiar (read: easy to learn) or provide such a major benefit (e.g. ES Promises) that developers are willing to invest the time to learn the new concepts. I fear most developers won't consider accessibility such a major benefit to learn a totally new API that entails a double-leap "source" abstraction of an "accessibilityElement" abstraction of Element.

but it's potentially more foolproof, it's more compatible with the idea of explicitly constructing virtual trees. My alternative idea (2) is simpler to understand but potentially requires the author to be more careful when implementing virtual accessible nodes.

Thoughts? Alternative ideas?

Option #3

Background: The DOMString role attribute is an ordered token list, so you can specify fallbacks, e.g. "switch checkbox" where the browser would pick either "switch" as the computed role if it had implemented the ARIA 1.1 feature, or otherwise fall back to the ARIA 1.0 "checkbox" role.

Web precedent: The DOMString className property/attribute is a readwrite string of whitespace-separated classname tokens. Most JavaScript frameworks implement convenience methods like addClass()/removeClass() so authors can avoid error-prone string manipulation. HTML recently added a readonly classList property of type DOMTokenList, with accessor methods: add, remove, toggle, etc. https://developer.mozilla.org/en-US/docs/Web/API/Element/classList

Proposal:

The role property remains immutable, but includes a setter method. Not particular to any name or syntax:

axElement.setRole("button");
axElement.role.set("button");
axElement.set("role", "button");

The method would return true if the setter was successful, and false otherwise.

Example reasons for setter failures include:

axElement.setRole("foo"); // unknown, unimplemented, or invalid role
axElement.setRole("option"); // invalid context dependency, option may be orphaned if not contained in a listbox

Regarding partial object overrides ("I want to define role, but let the UA return the label."), I feel like we can achieve this in a way that is more seamless to the author. A custom instance of AccessibilityElement could do the equivalent of calling super in the native code if this.hasOwnProperty() returns false.

customButton.accessibleElement = function(){
    return new AccessibilityElement({
        get role() { return 'button' },
    });
};

customButton.accessibleElement().role // calls the web author's getter
customButton.accessibleElement().label // non defined in the instance, so calls the native getter

Cheers, James

alice commented 8 years ago

(from Dominic)

The extra level of indirection with an accessible source seems like it would make adoption more confusing.

Agreed.

Wouldn’t we implement default values for many properties. i.e.) your accessibleElement.role would inherit role from the DOM element but you could override that if desired. More importantly, parent and children nodes would come for free unless you overrode with

button.accessibleElement.accessibilityChildren = [ ]

where you could then create an entirely fake accessible tree.

Yes, that's what I had in mind. What's not as clear to me is what would happen if you said button.accessibleElement = MyObject - are you allowed to pass an incomplete implementation in MyObject or not?

alice commented 8 years ago

(from Chris)

One way it could work is that we use the values from the incomplete MyObject where they exist, but then fallback to the DOM element to answer the rest of the questions.

alice commented 8 years ago

(from Dominic)

Have you considered an accessor instead of a property? This might fit better with your "instantiate accessibility only on demand" use case, and could also fit will well with the plan for virtual subtrees. For example, standard DOM elements would get element.accessibleElement() provided by the browser, but a canvas app author could overwrite the native method:

myCanvasElement.accessibleElement = function() { return myCustomAccessibilityViewController; }

This was the main goal behind our previously mentioned "accessibility delegate" idea:

Is there much of a practical difference? Any property can have a getter and setter if you want, and the property syntax seems like it would make the easy cases easier.

  1. Changing just one or two accessibility attributes on an existing element would be the easiest
  2. Adding a new virtual accessibility object that's always in the tree would be the next easiest
  3. Creating the accessibility object "on demand" would still be possible, using a getter, but would be more advanced.

One advantage to this is that doing this type of thing on-demand is harder to get right. For example, if an attribute of an object changes (like its "checked" state), an event needs to fire. We could do that automatically if you explicitly set that state on an accessible object - but if it's a getter that computes the state automatically it'd be up to the author to fire that event. So I'd prefer we usher developers towards building persistent objects if at all possible.

Minor nit: name should be label.

Makes sense.

No objection, but I'm curious what this should return in cases where there is not a 1:1 match with an ARIA role. For example, some custom web component, or the native <video> element? Should we consider the role property is limited to those defined by ARIA? I think so, but it leaves us with a lot of gaps for WebDriver. We could consider exposing the user agent's internal role as another property or accessor.

Good question. One idea: how about taking advantage of the fact that the ARIA role attribute can take on multiple values and allow clearly marked vendor extensions? For example, for the video element, WebKit could return "x-webkit-video group", or just "x-video group".

Finally if you wanted to replace all of the functionality of accessibleElement you could do that:

customButton.accessibleElement = {
  get role() { return 'button' },
  get name() { return 'name' },
}

This is similar to the accessor method.

customButton.accessibleElement = function(){
    return {
        get role() { return 'button' },
        get name() { return 'name' },
    };
};

One big difference here is that your version returns a different object each time. That could cause a lot of performance problems, and might trigger things like live regions inadvertently.

One benefit to the method over the property is that developers won't inadvertently trigger execution of accessibility code just by inspecting the properties of the containing Element. (Is there another way to keep the property variant from incurring the performance hit when the parent element is inspected?)

That's a great point, but I think that'd be manageable. In Blink we already addressed this by allowing accessibility to be enabled temporarily within a scope (we have a ScopedAXObjectCache). We use this for Chrome's developer tools too - accessibility code is temporarily enabled to compute those values and then it's all cleaned up.

In general I think it'd be a good thing if all browsers were able to better handle working with a few accessibility objects without turning on a full accessibility mode.

Option #3

Background: The DOMString role attribute is an ordered token list, so you can specify fallbacks, e.g. "switch checkbox" where the browser would pick either "switch" as the computed role if it had implemented the ARIA 1.1 feature, or otherwise fall back to the ARIA 1.0 "checkbox" role.

Web precedent: The DOMString className property/attribute is a readwrite string of whitespace-separated classname tokens. Most JavaScript frameworks implement convenience methods like addClass()/removeClass() so authors can avoid error-prone string manipulation. HTML recently added a readonly classList property of type DOMTokenList, with accessor methods: add, remove, toggle, etc. https://developer.mozilla.org/en-US/docs/Web/API/Element/classList

Proposal:

The role property remains immutable, but includes a setter method. Not particular to any name or syntax:

axElement.setRole("button");
axElement.role.set("button");
axElement.set("role", "button");

Makes sense.

The method would return true if the setter was successful, and false otherwise.

Example reasons for setter failures include:

axElement.setRole("foo"); // unknown, unimplemented, or invalid role
axElement.setRole("option"); // invalid context dependency, option may be orphaned if not contained in a listbox

I like all of this, but I think one reasonable option would be to allow it to be mutable but still have it fail if you try to set it to an illegal value.

As precedent for that, <input> behaves this way now - for example, try <input type="week"> and then try to set inputElement.value = "xyz" - if you query inputElement.value again you'll get the empty string.

Regarding partial object overrides ("I want to define role, but let the UA return the label."), I feel like we can achieve this in a way that is more seamless to the author. A custom instance of AccessibilityElement could do the equivalent of calling super in the native code if this.hasOwnProperty() returns false.

customButton.accessibleElement = function(){
    return new AccessibilityElement({
        get role() { return 'button' },
    });
};

customButton.accessibleElement().role // calls the web author's getter
customButton.accessibleElement().label // non defined in the instance, so calls the native getter

Yes, I like that but I'd prefer something like this:

customButton.accessibleElement = new AccessibilityElement({
  get role() { return 'button' },
});

Or even better:

customButton.accessibleElement = {
  __proto__: AccessibilityElement,
  get role() { return 'button' },
};

One nice thing about this second approach is that it'd be possible to define your own prototype that extends AccessibilityElement and reuse it for a bunch of objects, like all of the items in a list.

If the author tried to set customButton.accessibleElement to an object that didn't implement the full protocol it'd be an immediate runtime error.

minorninth commented 8 years ago

After discussing this with some more people, I have another idea that combines some elements of multiple ideas.

Option #4

The idea behind this is that there's an AccessibleSource, but it's more implicit and hidden for most use cases.

One problem with option #2 is that allowing the author to set the accessibleElement of an HTML element to any arbitrary object puts a lot of responsibility on the author to implement the whole protocol correctly. Instead, let the author provide any object, but it has to be wrapped in an AccessibleElement.

We start with Mozilla's idea but add a constructor.

interface AccessibleElement {
             AccessibleElement(AccessibleSource source);
             attribute DOMString          role;
             attribute DOMString          label;
             attribute AccessibleSource?  source;
};

This also makes it more straightforward to implement. An AccessibleElement is always a native-backed object. An AccessibleSource, if present, is an overlay that exists only in JavaScript-space, that provides a partial implementation of the AccessibleElement protocol. Any time the source is missing or invalid, it falls back on the default AccessibleElement implementation.

Furthermore, let's implement the setters for all of AccessibleElement's interfaces in a convenient but well-defined way. In particular, setting AccessibleElement.role has the effect of modifying the element's source, creating it if it didn't exist.

Let's look at some examples of how this would work. First, initially the source is just an empty object.

var customButton = document.querySelector('#customButton');
var axCustomButton = element.accessibleElement;
console.log(axCustomButton.role);  // prints "div"
console.log(axCustomButton.source.role);  // prints undefined

Setting the role actually modifies the source:

axCustomButton.role = 'button';
console.log(axCustomButton.role);  // prints "button"
console.log(axCustomButton.source.role);  // prints "button"

Things get interesting when you try to set the role to something invalid. If you do this, the source property takes on the value you pass, but the AccessibleElement property does not:

axCustomButton.role = 'foo';
console.log(axCustomButton.role);  // prints "div"
console.log(axCustomButton.source.role);  // prints "foo"

If you want to supply a role dynamically, you have to update the source. That's okay because it's a more advanced use case:

Object.defineProperty(axCustomButton.source, 'role', {
  get: function() {
    return 'button';
  }
});

Finally if you wanted to replace all of the functionality of accessibleElement you could replace the whole source:

axCustomButton.source = {
  get role() { return 'button' },
  get label() { return 'label' },
}

Alternatively you could construct a new accessible element, for use in a virtual hierarchy, by providing a source in the constructor:

var virtualAccessibleElement = new AccessibleElement({
  get role() { return 'button' },
  get label() { return 'label' },
});

In terms of implementation on the browser side, this becomes a lot more straightforward. an AccessibleElement is always a DOM wrapper for a native accessible object. They're always coupled; every native accessible object has exactly one DOM wrapper AccessibleElement, and every AccessibleElement has exactly one native accessible object for it.

An AccessibleElement always has a source, but it's initially just an empty object. The author can supply whatever they want, and if they supply something invalid or bogus, it doesn't "break" the native accessible object or the DOM wrapper.

From an author point of view, this makes the easy cases easy. If you want to modify an accessibility property, or even create a virtual hierarchy, you don't need to know what an AccessibleSource is. Only if you want to use getters or do something more advanced do you need to care about it.

alice commented 8 years ago

I wonder whether we even want to have two different flavours of AccessibleElement to account for the difference between a 'virtual' a11y node and a DOM-backed node?

interface AccessibleElement {
             attribute DOMString          role;
             attribute DOMString          label;
             attribute AccessibleSource   source;
};

interface VirtualAccessibleElement : AccessibleElement {
             VirtualAccessibleElement(AccessibleSource source);
             attribute DOMString          role;
             attribute DOMString          label;
             attribute AccessibleSource   source;
};

interface DOMBackedAccessibleElement : AccessibleElement {
             attribute Node               node;   
}

Then we could loosen up the restrictions on modifying VirtualAccessibleElement potentially? I guess it's tricky because either way it will need to be backed by a browser object and need "initial" values for all of its default (not pattern dependent) attributes.

Another idea: avoid having a constructor at all, but make a factory method like document.createElement():

var customTable = document.querySelector('#customTable');
var axCustomTable = customTable.accessibleElement;
var axCell = document.createVirtualAccessibleElement(new VirtualCellTemplate(1, 1, "Revenue"));
axCustomTable.appendChild(axCell);
// or
axCustomTable.insertBefore(axCustomTable.firstChild, axCell);
// etc...

I was also considering suggesting making it a factory method directly on the parent AccessibleElement, like Element.createShadowRoot(), but got stuck considering how to define ordering in that context.

Anyway - none of these thoughts are fully formed, sorry, but wanted to capture them while I was thinking about it.

minorninth commented 8 years ago

More details on AccessibleSource

In WebIDL we can make AccessibleSource a dictionary instead of an interface. That makes it clear that it's duck-typed, you can pass any object as an AccessibleSource if you want.

interface AccessibleElement {
             attribute DOMString          role;
             attribute DOMString          label;
    ...
    readonly attribute AccessibleElement? parent;
    readonly attribute AccessibleElement? firstChild;
    ...
             attribute AccessibleSource?  source;
};

dictionary AccessibleSource {
    DOMString role;
    DOMString label;
};

I'm still really liking the idea that you can try to modify attributes of an AccessibleElement directly, but in doing so you're really just modifying the AccessibleSource overlay. That leads to really clear semantics:

  1. Try setting the role to something experimental. If it fails, you know the browser doesn't know about that role, but the string you passed in is still in AccessibleElement.source.role
  2. Clear all modifications I made to an element: element.accessibleElement.source = null; - that reverts it to getting its accessibility from the DOM.

Order of precedence

AccessibleElement should reflect what's actually reported to the native accessibility API.

AccessibleSource would be the highest priority, if set and if valid.

ARIA would be the next highest priority.

The DOM / would be the last priority.

Setters and ARIA reflection

I still like the idea that setting a property on AccessibleElement is a shortcut for setting the corresponding property in AccessibleSource. That makes common cases simpler and less verbose, while allowing for some more advanced use cases.

Another idea would be for the setters on AccessibleElement to modify ARIA attributes too.

For example, suppose you have this element you want to make into a checkbox:

<div id="subscribe" tabIndex=0>
  I want to receive spam
</div>

You could set its role and checked state from JavaScript like this:

$('subscribe').accessibleElement.role = 'checkbox';
$('subscribe').accessibleElement.checked = false;

This could actually modify the HTML, where possible, so you'd get this:

<div id="subscribe" tabIndex=0 role="checkbox" aria-checked="false">
  I want to receive spam
</div>

However, if you do not wish to modify the HTML, this lower-level call would have the same effect without modifying the HTML:

$('subscribe').accessibleElement.source = {
  role: 'checkbox',
  checked: false
};

This might be a nice balance between keeping accessibility as declarative as possible, while allowing for more advanced use cases where making it declarative is impossible or undesirable.

alice commented 8 years ago

You could set its role and checked state from JavaScript like this:

$('subscribe').accessibleElement.role = 'checkbox';
$('subscribe').accessibleElement.checked = false;

This could actually modify the HTML, where possible, so you'd get this:

<div id="subscribe" tabIndex=0 role="checkbox" aria-checked="false">
  I want to receive spam
</div>

This might be a nice balance between keeping accessibility as declarative as possible, while allowing for more advanced use cases where making it declarative is impossible or undesirable.

I'm not convinced keeping accessibility declarative should be a goal. I think I would rather setting accessibleElement.checked not set the ARIA attribute (but perhaps clear any attribute set if there was one, to avoid confusion).

I realise it's possible to avoid sprouting attributes via the mechanism you described, but I feel like having the attributes sprout by default is going to surprise and annoy people, and it's non-obvious how to get around it.

cookiecrook commented 8 years ago

Today's meeting outcome, as I understand it.

  1. Everyone agrees we need to add a few extra properties/accessors/etc to DOM-backed views. This should be dead simple (no a11y source necessary?) for the 99% of web developers that use standard DOM-based views but may need a little something more. (e.g. Is a11y ignored for this element? If not, what is its computed role?)
  2. Most (except perhaps Microsoft?) agree we need a way to expose virtual accessibility trees:
    • For complete virtual trees, build up the tree all at once
    • For large collections that are incomplete in the view (spread sheet tables, feed collections, etc), there could be a way for AT to access some data from the local view model (e.g. get the contents for cell at row/col index, even if that cell is not rendered in the view)
  3. [Raising as new Issue #10] There is a shared desire for additive behaviors, taking the form of multiple proposals: control patterns, delegate protocols, mixins, etc.
  4. Should this continue in a Working Group, Community Group, Incubator Group, or some other non-W3C standards body? Undecided. Still more work to be done within the GitHub group to agree on a shared proposal. We can worry about the formalities once we have a basic plan of action.
cookiecrook commented 8 years ago

Today's action: Dominic and Alex to attempt some polyfill-type implementations of simple test cases using the following properties:

And a single action of "click"

cookiecrook commented 8 years ago

Action: Everyone to add sample test cases (within 1.0 scope) to Issue #12