Accessibility of model

[marcoscaceres]

We need to figure out how to make <model> accessible on a number of different fronts:

• Visual: describe what is being presented over time.
• Interaction: describe what can interacted with (regions, buttons, etc.).
• auditory: describe audio and possibly spoke sounds, potentially over time.

Usually, this would be provide by the embedded format... however, it appears that both glTF and USDZ are quite limited when it comes to accessibility.

As such, it may be that we need to leverage what we can from HTML + ARIA to overcome the shortcomings of these formats. We have quite a bit of precedent (e.g., from the humble, yet limited, alt attribute, to how <canvas> can be made accessibly, to the potential inclusion of <track> elements, and so on).

[klausw]

Side note, the list above is of course not exhaustive. There are additional accessibility considerations apart from adding descriptions, for example support for alternate input methods such as keyboard navigation or voice-activated controls. Also, some accessibility tools on touchscreen devices change how interactions work, for example requiring two- or three-finger gestures for rotation or zooming.

[marcoscaceres]

Oh yes, totally. No doubt there is ton of stuff missing. We need to go over the papers from the inclusive xr workshop and bring in the idiw folks.

[cabanier]

Usually, this would be provide by the embedded format... however, it appears that both glTF and USDZ are quite limited when it comes to accessibility.

That is very unfortunate :-\ Can we file issues on both formats so this can be added?

As such, it may be that we need to leverage what we can from HTML + ARIA to overcome the shortcomings of these formats. We have quite a bit of precedent (e.g., from the humble, yet limited, alt attribute, to how <canvas> can be made accessibly, to the potential inclusion of <track> elements, and so on).

I agree that that's the right approach. There needs to be a way to translate a model into an accessibility DOM so it can be exposed to screen readers.

[donmccurdy]

The nearest feature I'm aware of in glTF is support for the XMP packets is JSON-LD format, including Dublin Core, attached to individual nodes within the model's scene graph. These may include titles, descriptions, and other semantic metadata about that particular part of the model, comparable to "alt" text.

It is not obvious to me that we want every model format inventing its own accessibility standards. I wonder if this could be done at a higher level, through a W3C specification, an XMP namespace, or something else. This would probably have value even outside of the <model/> proposal, as I'm not aware of any standard 3D file formats or authoring tools working on this today. There has been some (very early) work to add namespaces to XMP relevant to 3D models, for things like alignment to vertical or horizonal planes in AR.