What are the intended effects of the math role and what type of guidance should be in the APG?

[ZoeBijl]

From @pkra on March 9, 2016 10:1

Over on a11ySlackers I stumbled upon http://w3c.github.io/aria/practices/aria-practices.html#math again which always leaves me somewhat confused. Since I'm only a beginner, I was hesitant to post a confused question but @MichielBijl encouraged me to do so, so here I go.

Specifically, I don't understand the example 43 in the Practices WD.

Shortened version:

<div role="math" aria-label="a times x squared plus b times x plus c equals 0">
  <math xmlns='http://www.w3.org/1998/Math/MathML'>
  <!-- snip -->
  </math>
</div>

Since I'm only a beginner, I was wondering what the expected result of this is.

At first I thought this should lead to double-rendering by MathML-capable AT (first the label, then the MathML). However, most ATs I tested simply ignore anything with role=math so the label will not be read (or anything else). If I drop role=math, then NVDA(+MathPlayer) will voice both the label and the MahtML in Firefox.

Then I thought the label would override the remaining content (e.g., that's what Chrome's a11y dev tool extension indicates), but that would mean that AT with MathML support is forced to use the lower-quality option. So I'm probably wrong about that, too.

To give some context, I don't care too much about the MathML, actually. The real problem I see is to make HTML renderings of mathematics accessible and neither the Practices nor the math role description help much with this.

For as simple an example as the one in the Practices, "deep" labels provide quite decent accessibility, e.g.,

 <math>
    <mrow>
      <mrow>
         <mrow>
            <mi>a</mi>
            <mo aria-label="times">&#x2062; <!-- invisible times --></mo>
            <msup aria-label="x squared">
              <mi aria-hidden="true">x</mi>
              <mn aria-hidden="true">2</mn>
            </msup>
         </mrow>
         <mo>+</mo>
         <mrow>
            <mi>b</mi>
            <mo aria-label="times">&#x2062; <!-- invisible times --></mo>
            <mi>x</mi>
         </mrow>
         <mo>+</mo>
         <mi>c</mi>
      </mrow>
      <mo>=</mo>
      <mn>0</mn>
    </mrow>
  </math>

would seem to make more sense to me. But due to the absence of MathML support almost everywhere in the a11y stack, this doesn't actually work. (I would point out that the label actually enhances the original MathML as superscripts carry no semantic meaning in Presentation MathML either.)

However, if one follows the suggestion to use a polyfill MathML-to-HTML5 converter, then there's actually a good chance this could become more accessible than a fixed label. Cf. http://codepen.io/pkra/pen/zqrwQx: the third example renders nicely on NVDA, even allowing some level of exploration. Just to be clear, this only works well for relatively "flat" equations like this one but that likely covers a worthwhile amount of content, especially inline equations.

And one last, related question regarding the "generic HTML" example in the Practices WD. Why can we not expect AT to render this as well as it would render the MathML counterpart, once we add a role=math to it?

<span role="math"><i>a</i><i>x</i><sup>2</sup> + <i>b</i><i>x</i> + <i>c</i> = 0</span>

There's not really a semantic difference between msup/msub and sup/sub, albeit a difficulty of parsing of sup/sub; but that is something Practices can actually help with by recommending wrapping, e.g.,

<span role="math"><i>a</i><span><i>x</i><sup>2</sup></span> + <i>b</i><i>x</i> + <i>c</i> = 0</span>

Not that this works today, but I'm wondering why it shouldn't be. (But I realize this leads to the larger problem of role=math and the lack of ARIA roles for mathematics so I better stop here.)

Copied from original issue: w3c/aria#290

[ZoeBijl]

Thank you for your elaborate issue report! What I take from it is that the current example isn't the most accessible. So is your suggestion we replace the example with something else? Your second example looks good to me, but I don't know much about maths so can't really tell what it should convey.

[ZoeBijl]

From @pkra on March 15, 2016 15:12

So is your suggestion we replace the example with something else?

I don't know. My initial question was: What is the expected result of the example in the spec?

AT is inconsistent in what it voices (sometimes the label, sometimes the MathML, sometimes both sometimes neither); it's not clear to me what the expected result should be so it's hard to report issues with AT (let alone create reliable content).

Your second example looks good to me,

Thanks, but it works in even fewer AT combinations than the original example from the spec. Again, I'm not sure if that's expected behavior or not.

Let me stop here, perhaps, before adding too much noise.

[ZoeBijl]

I can bring this up during the ARIA or APG calls. Will report back once discussed.

[ZoeBijl]

From @pkra on March 15, 2016 15:33

Thanks, @MichielBijl. I'm happy to provide more input if that helps.

[ZoeBijl]

From @pkra on March 21, 2016 10:37

FYI I've updated the codepen at http://codepen.io/pkra/pen/zqrwQx and added test results (not completely done but as far as I'll get for now; additions are welcome). The results are rather inconsistent.

[ZoeBijl]

From @laughinghan on April 11, 2016 19:55

I'm also a novice in this area and I'm not the main accessibility developer on my project but for us, the equivalent of:

<span class="we-need-a-container-anyway" aria-label="x superscript 2 baseline y"><span aria-hidden="true">x<sup>2</sup>y</span></span>

has been working in the screen readers we've been testing (I've been testing with VoiceOver and Safari; I believe our main accessibility developer tests mostly with WindowEyes but also with JAWS and NVDA.

Is that surprising to anyone? Does that sound like a bad approach?

[pkra]

Thanks for moving this here.

Does that sound like a bad approach?

I had meant to respond to that on the other thread earlier but got side-tracked -- sorry.

I think most people would say that long alt-text strings / labels are a bad approach. But it's the only reliable one -- no criticism to @laughinghan or the MathQuill team!

The main reason why this is problematic is that math expressions can easily require enormous strings. So we run into the problem of exploration methods for alt-text as well as the limitation of offering just one alternative rendering.

As I showed in my codepen examples, I think what we should look for is "deep" labels, which enable some level of exploration (though I'm not sure if that's unintentional -- input would be nice). However, deep labls are not always possible today (not just practically speaking from existing AT but also structurally since visual rendering and semantic information can be highly incongruous).

What I'd love to see in the future is some ARIA functionality that allows a content creator to provide sensible fragments at varying levels in a way that allows AT to easily(!) build multiple representations, e.g., depending on user options. That's not as hard as it sounds, I think; it certainly seems simpler than expecting all AT to magically enrich mathematical rendering with semantic information. It's also closely related to the problems of SVG accessibility where top-level descriptions ("a red house" and detailed descriptions of parts ("a door with window") form a similar, though less structured example of this problem. Looking at Han's example you see the typical MathSpeak "baseline" thing. MathSpeak actually has three modes for various degrees of verbosity, so you'd really like to expose parts like "baseline" with an indication that it's optional for AT. Similarly, in a longer expression, you might provide a high-level overview over the expression ("a summation equal to an integral") on the top level while allowing for deeper exploration to get to the actual content.

But now I'm back in proper ARIA land (not Practices), I suppose. I hope ARIA will be open to discussing these problems with people in the newly formed Math-on-web-pages CG. As I wrote on a11y slackers, I'd be happy to join a call some time.

[laughinghan]

I see what you're saying about the enormous strings and exploration, the large aria-label exposes the content but obscures the structure from machine-readability.

Does ARIA give authors a way to specify ordering? Like, for math for example, is something like:

x <sup aria-before-label="superscript" aria-after-label="baseline">2</sup> y

(which is still intended to be read "x superscript 2 baseline y") even possible?

There are even more troublesome cases, like summation notation where the document order won't match the desired read-order:

<span class="operator-container" aria-label="sum from n=1 to 10 baseline x sup n">
  <span class="upper-limit">10</span>
  <span class="operator">&sum;</span>
  <span class="lower-limit">n=1</span>
</span>
x
<sup>n</sup>

How could we get the lower limit to be read before the upper limit?

[pkra]

There are even more troublesome cases, like summation notation where the document order won't match the desired read-order.

Right -- and many more, and worse than that :wink:.

How could we get the lower limit to be read before the upper limit?

There's aria-flowto but I don't know how well it works in the wild.

[pkra]

I just wanted to follow up on this. Given https://www.w3.org/2016/04/11-aria-apg-minutes.html#item03, I was hoping we could set up a call between interested folks from WAI/ARIA land as well as from the newly formed Math On Web Pages CG to discuss issues surrounding math accessiblity and perhaps even identify future work the CG could help with.

[ZoeBijl]

Pinging @jongund for the call mentioned by @pkra.

[pkra]

FWIW, still interested.

[jnurthen]

@pkra if you have any concrete edits you would like to propose for this issue in the APG we would very much appreciate it. None of us in the APG are Math experts so we would really appreciate help.

[pkra]

Thanks for the reply, @jnurthen! I'd be happy to contribute something but I'm still not sure I understand the specs correctly.

If I may go back to my initial question (which really has nothing to do with math, I promise :-) ):

What is the expected result of the example from the document? That is,

<div role="math" aria-label="a times x squared plus b times x plus c equals 0">
  <math xmlns='http://www.w3.org/1998/Math/MathML'>
  <!-- snip -->
  </math>
</div>

As I wrote initially, it seems to me that this should lead to duplicate voicing: first, the label on the wrapper is voiced, then the MathML is voiced.

Is that a correct interpretation of the spec? (Ignoring implementations and their behavior for a minute.)

[clapierre]

if aria-label is problematic due to size constraints of enormous strings. Wouldn't a better idea then would be to use the new ARIA Details/Summary that is being proposed in ARIA 1.1? Here is an issue that the WAI APA is currently discussing on making the ARIA Details/Summary discoverable by AT as well as adding the ability to hide it if the user so chooses. https://github.com/w3c/html/issues/561

[jnurthen]

@pkra sorry for the delay responding. No - what you have written should not lead to duplicate voicing (although in practice I have no idea what happens). role=math has children presentational set to true so in effect the div with role=math will have no child nodes in the accessible tree.

[clapierre]

The IDPF currently has a techniques document written on how to write MathML in EPUB 3.1 which now looks to be incorrect after reviewing this issue. http://www.idpf.org/epub/a11y/techniques/techniques.html#sec-desc-002 There will be a call to discuss this issue where we would love to get expert opinions on what would be the best way to recommend to publishers on how to include math in their publications.

The meeting will be held on Wednesday August 24 @ 16:00 UTC Please email charlesl[at]benetech[dot]org if you would like to attend and I will provide you with a link to the dialing information. We will be reaching out to the MathJax community as well as Math on the Web Community Group.

Thank you Charles LaPierre Benetech co-chair of the Accessibility Task Force for the IDPF's EPUB 3.1 specification

[pkra]

@jnurthen thanks so much for this clarification!

role=math has children presentational set to true

This blows my mind. But at least I now know what the desired behavior is.

[pkra]

@clapierre

if aria-label is problematic due to size constraints of enormous strings.

I'm personally not concerned with this. If you check out the examples in my codepen link above, I'm interested in getting "deep" labels to work. Those tend to be very short.

Wouldn't a better idea then would be to use the new ARIA Details/Summary that is being proposed in ARIA 1.1?

See above but on a maybe related note: I was recently asking on a11y slackers about providing summary/overview information for the subtree below a node while preserving the nodes (think of this iteratively) and @MichielBijl pointed me to aria-details. I haven't had a chance to really dive into it but it seems that aria-details would always overrule other content (e.g., -describedby, -label), so it doesn't work for providing alternatives such as summaries.

The IDPF currently has a techniques document written on how to write MathML in EPUB 3.1 which now looks to be incorrect after reviewing this issue.

My issue at https://github.com/IDPF/epub-revision/issues/681 might be relevant.