Adding roles, states, and properties to support math

[dani31415]

The need for the role parity (for each off the MathML tags like msub, mrow, ...) is that MathML markup is rerely used for the actual displaying of mathematics. In fact, most of the time, formulas are represented as:

• SVG
• HTML created with tools like MathJax/WIRIS

Another benefit is that it will probably make it easier to implement mathematics by AT if they have to use roles instead of navigating and interpreting the MathML tags.

Currently most publishers write formulas using MathML and convert them to any other format at delivery time. We would like to preserver the MathML original structure as much as possible independently of the output format.

Let’s remember that currently only Firefox implements MathML natively despite the efforts to implement it in other browsers.

[pkra]

Related discussions: https://github.com/w3c/aria/issues/425

[joanmarie]

Actually it's also implemented in WebKit (Safari and WebKitGtk). Hypothetically, if MathML were implemented natively in Chrome and Edge, would that solve the problems?

[dani31415]

It won't solve the problems with formulas that are inside an SVG as part of a graph.

That all browsers implement MathML does not necessary mean that all math have to be done with MathML. For exmaple, HTML has native buttons but many developers decide to implemented them with images and JavaScript.

There is the issue also about how long it will take MathML to be implemented.

If someone argues that there are a lot of MathML tags, why not use custom roles or sub-roles? (but this might be an idea for another issue).

[asurkov]

I see the point of having ARIA roles for math, for same reasons we have roles for HTML: people will probably always use non standard ways to represent data.

[pkra]

Hypothetically, if MathML were implemented natively in Chrome and Edge, would that solve the problems?

No, MathML is too limited.

But I think this issue is covering the wrong topic.

First off, on the web MathML only means Presentation MathML; Content MathML does not exist in significant qualities in the wild and even less so on the web.

However, Presentation MathML does not actually encode semantic information, it encodes layout information (for what is historically called equation or formula layout). As such, Presentation MathML is used to encode formula layout from various scientific fields with an enormous range of semantics. Furthermore, Presentation MathML is deeply rooted in print layout which often poses problems on the web, e.g., the abundance of tabular layout, harcoded font variants etc. To be clear, I'm not trying to bash Presentation MathML; after being kicked out of HTML 3, its success was in XML-to-print workflows and it does a great job there.

Even if one considers the semantic information generated from Presentation MathML using heuristics (e.g., via speech-rule-engine), Presentation MathML is too limited to express many mathematical concepts (e.g,. simple commutative diagrams). It is also limited by focusing on single expressions without (reasonable) integration into larger document structures which causes issues in many basic educational settings (e.g., explanatory diagrams in elementary school books are usually impossible).

In short, having Presentation MathML's layout information encoded in ARIA seem fairly useless to me.

Having some actual mathematical concepts in ARIA (e.g., "number" or "fraction") seems useful (but also much harder since there is no spec to align with).

Identifying "mathematics" with "Presentation MathML" would (and actually does) significant damage (on the web and elsewhere).

[joanmarie]

Next steps: People with math expertise should start to identify what roles, states, and properties are needed. I will help with things needed to make it ARIAy and specy. :)

[asurkov]

@joanmarie is there a pool where suggestions should be hosted?

[cookiecrook]

Perhaps there should be an "accessible math" incubator in WICG?

[cookiecrook]

As far as I know the two options now are:

• Accessible MathML (only supported on Apple platforms)
• MathJax (good cross-browser support for voice and keyboard nav, but implemented as a self voicing polyfill which prevents native Nemeth braille support provided by WebKit's MathML implementation)

[kevinbarabash]

Support both semantic and presentational would be useful. Semantic representations feel most useful for consuming content whereas presentational could come in hand for interactive applications as well as authoring content on the web.

[cookiecrook]

Apple platforms have retained MathML support (inc/Nemeth) for about a decade, and other platforms have better MathML support too.

Furthermore, MathJax (version 2 or 3?) no longer removes the underlying MathML structure, so a user gets to choose between native formula exploration in VoiceOver, or MathJax's formula exploration, and VoiceOver's Nemeth braille support is retained too. I've recently become aware of an NVDA extension that leverages MathJax and perhaps MathML support, too.

[NSoiffer]

Minor correction and amplication to what @cookiecrook wrote: NVDA via the MathPlayer addon has supported MathML for at least a decade. MathPlayer is end-of-life, so I started work on MathCAT a few years ago. It too is an NVDA addon, but NVDA plans to incorporate it directly into the source this year so that, out of the box, NVDA will read math.

MathJax has had an option to include hidden (off screen) MathML in its output. Originally it was an opt-in option. But recognizing the importance of accessibility, many years ago it became opt-out. This allows NVDA, Jaws, ORCA, and VoiceOver to read math rendered by MathJax. Wikipedia uses a server-based version of MathJax that includes the hidden MathML -- MathJax is not running on the page downloads from Wikipedia.

Finally, I suspect most readers are aware of this, but just in case they aren't: as of Jan 2023, all major browsers support MathML natively. It would be great to get the underlying OS accessibility APIs to have something for math to map to, similar to what Apple has done. This doesn't have to be a clone of MathML, but it should have some way of mapping basic math notations.

[cookiecrook]

The consensus from the ARIA F2F was that we may need a MathML-AAM, Ruby-AAM, and others. In addition to platform mappings, the computedrole string for WPT testability would be more or less one-for-one roles like <mfrac> -> computedrole -> "mathml-mfrac" according to discussion re: https://github.com/w3c/aria/issues/529#issuecomment-1539101046 and related issues.

[cookiecrook]

Which could allow implementations to map other math technologies like LaTeX, but would not allow authors to turn tag soup into MathML... E.g. <div role="mathml-mfrac"> would not work.

[jnurthen]

this is unlikely to get consensus so closing.