Expose MathML for accessibility

[jcsteh]

In order for screen readers and other assistive technology (AT) products to provide rich access to math content, they need to be able to access the raw math in a standard way. Providing alternate text is not sufficient, since although this does allow for better spoken math, it does not provide for braille, interactive exploration, etc. It seems MathML has become the dominant standard for exposing math content and MathJax uses MathML internally, so it'd make sense to expose it as MathML.

You can already use the toMathML extension to access MathML for a MathJax tree. The problem is that this requires JavaScript calls and an AT does not always have the ability to execute arbitrary JavaScript on the page. (For example, in Firefox, this isn't possible without a browser extension or Greasemonkey script.) There needs to be some way to obtain MathML from MathJax that does not require JavaScript calls.

One solution is to expose the MathML via an attribute; e.g. data-mathml. This is pretty inelegant and non-standard, but it does solve the problem. I've already implemented experimental support for this in NVDA's (not yet released) math support.

If exposing an attribute like this automatically is considered too expensive, we might be able to come up with a solution using an off-screen button or similar which exposes the attribute when activated. However, this is even less elegant and is more error prone, so I'd hope to avoid something like this if possible.

/cc @pkra

[jcsteh]

/cc @NSoiffer @camlorn

[jcsteh]

I created a Greasemonkey script to do this for MathJax on any page with the toMathML extension loaded. Unfortunately, this isn't working with latest Firefox + Scriptish because of security changes in Firefox. I'm not sure about Greasemonkey.

[pkra]

Thanks for creating the bug! I'm guessing this can start out as an extension and assuming no hidden issues show up, it could ship with the combined configuration files (which most people use).

As for the implementation, an easy (though somewhat strange) way would be to simply attach the data attribute it to the (internal) MathML -- thanks to #860 this will be preserved. Alternatively, the solution for #860 could be re-used.

[pkra]

Related to #939, I guess another option should be to trigger this in via an (accessible) menu so that even in cases where someone custom build a configuration the user can trigger this.

[ahicks92]

At the moment, per-site, it's possible to turn it on. It takes something like 10 keystrokes and rather esoteric knowledge from the perspective of most users. I.e. of all the blind people I know, there's like 5 who would actually be able to follow my instructions, and one of them is @jcsteh. Definitely +1 to this.

[jcsteh]

I neglected to mention that when this exposure is enabled, the root of the MathJax output tree (where this attribute is exposed) should also expose the attribute role="math".

[pkra]

Thanks for the additional comments!

[NSoiffer]

Both Wikipedia and Kahn Academy expose MathML. For the record, here's what they do:

Wikipedia

After setting the math display option to MathML with SVG or PNG fallback

FireFox

class="mwe-math-mathml-inline mwe-math-mathml-a11y" style="display: none;"

.mwe-math-mathml-inline {
    display: inline !important;
}

.mwe-math-mathml-a11y {
    clip: auto;
    overflow: visible;
    position: static;
    width: auto;
    height: auto;
    opacity: 1;
}

Chrome

.mwe-math-mathml-inline {
    display: inline !important;
}

.mwe-math-mathml-a11y {
    clip: rect(1px,1px,1px,1px);
    overflow: hidden;
    position: absolute;
    width: 1px;
    height: 1px;
    opacity: 0;
}

Kahn Academy

The HTML-CSS ugliness that KaTeX generates has the aria-hidden attr on it. The MathML (class="katex-mathml") makes itself small so it isn't visible:

.katex .katex-mathml {
    position: absolute !important;
    clip: rect(1px, 1px, 1px, 1px);
    padding: 0px !important;
    border: 0px none !important;
    height: 1px !important;
    width: 1px !important;
    overflow: hidden;
}

[pkra]

Thanks so much for those details, Neil. As you can see, this issue is in the milestone for the next release.

However, the OP proposes a different solution -- exposing MathML in a data-attribute so that AT can pick it up more easily. My impression is that, at this point, embedding MathML in the DOM is really only necessary for VoiceOver (and at least Wikipedia has a bug report for VoiceOver not working with its embedding). IIRC, all other math-enabled screenreaders can pick up MathJax's internal MathML already, so we're hoping that covering these two use cases will be a good iteration.

In any case, we'll bring our proposal up for discussion here and on the dev-list and reach out to other AT vendors.

[NSoiffer]

I think the only AT that can pick up MJ's internal MathML is JAWS in IE only. IE is the only browser that makes calling JS not too difficult. If you embed the MathML in the DOM as opposed to "hiding" it or not exposing, then I think AT will "just work". E.g, when Kahn Academy did it, NVDA started working with it without any new work.

AFAIK, I've convinced the JAWS folks to look for $on\; a\; page\; because\; trying\; to\; get\; it\; to\; work\; in\; Firefox\; and\; Chrome\; was\; a\; lot\; of\; work\; and\; fragile.\; Having\; to\; do\; something\; special\; based\; on\; what\; technology\; is\; rendering\; math\; isn\text{'}t\; a\; great\; solution,\; so\; doing\; something\; that\; works\; independent\; of\; MathJax\; running\; or\; not\; would\; be\; best\; as\; it\; removes\; the\; need\; to\; do\; special\; casing.$

The one place where maybe you need to know about the renderer is for sync highlighting. But if AT could just tweak foreground and background of the mathml in the DOM and the renderer would pick that up, that might be best. That's a distant future though probably based on ShadowDOM. On the other hand, the interface we worked out with TextHELP is probably a lot easier to use for AT.

[sinabahram]

This is true.

In fact, I wrote a bookmarklet, MathInjector, to extract data-mathml and stuff it in a page with MathJax.

https://github.com/SinaBahram/MathInjector

I’d love to avoid ever having to use said bookmarklet e.g. AT does it automatically. There are some encoding issues, though (see comments in bookmarklet).

[jcsteh]

data-mathml is a massive hack. The only reason I suggested it was to avoid exposing real MathML in the DOM and having to visually hide it somehow, as well as having to hide the HTML-CSS rendering for accessibility. Real MathML would of course be much nicer, but see below.

There are two problems with these "hiding" solutions:

1. Because there's no explicit facility for having content visible to accessibility but hidden visually, different ATs may handle it differently.
2. Even if you find an approach that works for all ATs, ATs very possibly won't get the on-screen location of the math being visually displayed. For example, taking what Kahn Academy does, the AT will be told that the math has a width and height of 1 and the left and top coordinates might even be wrong too. This means, for example, that AT users won't be able to invoke the MathJax context menu. If the AT wants to display its own representation of the math for some reason, it won't know where to position it. I'm not sure about this one, but it may cause problems for magnifiers too, since they may not be able to get the location information they need.

I seem to recall hearing (but could well be wrong) that JAWS can only pick up MathML from MathJax if the input was MathML (and thus the MathML is hidden in the DOM); i.e. it won't work if the input was LaTeX. If that's the case, they're going to need a solution like this too to support other input formats.

[sinabahram]

I’m typing out loud here.

Regarding the first point, while it’s not an explicit thing, we’ve used off screen content for over two decades. I think there’s a nice combination of CSS that works across Jaws, NVDA, VoiceOver, within Firefox/IE/Chrome/Safari, etc.

I’ve not seen any AT not pick up on such things. At least, no AT that I’ve wanted to have access to it e.g. I obviously wouldn’t expect zoom text to pick up on it, and that’s a good thing, since it’s being hidden visually.

Regarding your second point, wouldn’t a reverse aria-describedby be a good idea here? we can put an aria-describedby on the image or other silly representation being chosen, and assign it’s value to the id of the off screen content. I only say silly because I think we’d all prefer that the MathML be the true representation, but unfortunately, given the lack of rigger in its visual presentation, certain mathematicians, I’m told, still prefer what they can achieve via LaTeX typesetting and other means.

There are accessible name calculations to think through, but if the accessible name of the mathematics is simply the output from running it through mathplayer, and or that can be assigned to it at runtime via an injection of aria-label onto said MathML, then the describedby pointing to it should pick that up and slap it onto the element being displayed visually. Since we’re doing an aria-describedby instead of an aria-labeledby, it won’t matter if MathPlayer isn’t installed, thus returning nothing from the aria-label of said MathML. E.g. graceful fallback to alt of the image.

Thoughts?

[pkra]

Thanks for all your comments -- we really appreciate it.

Re @NSoiffer

I think the only AT that can pick up MJ's internal MathML is JAWS in IE only.

IIRC, ChromeVox, Texthelp and GH all use MathJax directly in various settings (web and apps), including synchronized highlighting.

If you embed the MathML in the DOM as opposed to "hiding" it or not exposing, then I think AT will "just work".

From what I've heard, all hidden embeddings so far have had problems, so I'm a bit sceptical (also when ignoring the performance penalty for this).

Thanks also @jcsteh and @sinabahram for those valuable points. @sinabahram, aria-describedby is on our todo list to investigate; do you happen to know if there's a way to specify multiple alternatives? We can imagine a situation where we would provide MathML, TeX and speech text possibly more author-provided formats. (Also, we would want to expose the MathML tree structure in the output.)

The larger problem is that native brower support is not on the horizon and web components are also not very close to being usable in production (neither natively nor with polyfills, which IIUC also cause a11y problems).

So we think it would be a good idea to develop a practical and generic way of exposing the underlying data (i.e., MathML but potentially other source) in the rendering. In my eyes, this would ideally be something we could publish as a note of the MathWG.

The goal would be that all MathML renderers and AT solutions could adopt it, independently of any one rendering solution or any one AT math-component. It could even help accesibility solutions for other formats move forward (e.g., ChemML).

This does not seem unattainable to me.

Perhaps it makes sense to set up a call with interested folks as soon as possible. For those interested, drop me an email at peter.krautzberger@mathjax.org.

[jcsteh]

@sinabahram: While you may be correct about off-screen content in most cases, @pkra noted that there's a bug against Wikipedia regarding VoiceOver not detecting the MathML. Even if this is somehow incorrect, the point is that this is highly unreliable/error prone.

IMO, aria-describedby isn't an option here for several reasons:

1. An AT would still need to have explicit support to hunt for the node referenced by aria-describedby and handle it as math. It'd also need to map the on-screen location to the describedby node. If we're going to do that, we may as well just do data-mathml, which is less convoluted (if still hacky).
2. Even if an AT did this, the math would appear twice to the user, since the description (as referenced by aria-describedby) would still be present in its original place as well. (We don't "move" the description from its original place to the place where it is referenced for various reasons.)
3. Browsers/existing ATs would try to calculate text from the math referenced by aria-describedby, resulting in a very confusing description.
4. The output from MathPlayer can't simply be made the accessible name. It contains stuff other than text (pauses, etc.), which accessible name can't handle.

Basically, using aria-describedby is still a nasty hack, is more convoluted and doesn't solve any problems better than data-mathml. I'm very much opposed to this solution. :)

[sinabahram]

Great points, and thanks for making them.

So, it seems like data-mathml is the way to go, barring any other suggestions?

Should we agree on a standard encoding?

[pkra]

Here are some notes coming out of our last developer meeting. I'll send out an email to schedule a call. Not sure how to reconcile the time zones involved but we'll sort it out.

Some background.

• MathJax attaches ids to all elements of its output (for various reasons)
  • if the source includes an id (e.g., a MathML element), this id will be re-used for the corresponding element in the output.
• The new CommonHTML output can be highlighted directly, a highlighting extension for the HTML-CSS output is in the works.

Proposal(s)

• Structure of the exposed MathML
  • The exposed MathML will correspond to the internal MathML used to generate the output (not necessarily identical to the source)
  • Each MathML element will carry an id ( if preferred: a data-* attribute) corresponding to the id of the element corresponding to it in the output.
  • This allows AT to easily link the MathML structure with the output structure.
• Options for embedding
  1. Add a data-mathml attribute with serialization of the MathML (with ids)
     • mapping to output
     • AT will serialize anyway (true?)
     • simpler implementation, better performance
  2. Add hidden MathML directly (without ids)
     • MathML is in the DOM
     • accessible to DOM exploration
     • no mapping to rendering
     • rendering needs to be hidden (or aria-describedby used but as discussed this has other problems)
     • could be made to work with VoicOver
  3. Add hidden <object> element containing the MathML (with ids).
     • MathML in the DOM
     • accessible to DOM exploration
     • retaining a direct mapping via the ids (since <object> elements are a separate document there is no conflict).

The team is leaning towards (i.) but we want to implement what works best for AT (or more pricesly, its users).

[jcsteh]

Thanks for the great summary.

Regarding serialisation, NVDA does serialise, but it's possible that VoiceOver doesn't serialise into MathML, as it obtains math via the accessibility hierarchy. That may preclude data-mathml as an option if we want VoiceOver support. I guess someone will need to discuss this with Apple accessibility.

It's worth noting that visually hiding the MathML and hiding the HTML-CSS for accessibility would certainly work for both NVDA and VoiceOver right now. However, it does have the disadvantages I noted regarding associating to the visual location, etc. Primarily, this means no access to the MathJax context menu and no ability to display an overlay over the math. The former in particular is pretty serious.

[pkra]

Thanks, @jcsteh.

[...] That may preclude data-mathml as an option if we want VoiceOver support.

Right, we are not assuming data-mathml would work with VoiceOver; a specialized solution would probably be necessary (just like it is for MathPlayer 3).

I guess someone will need to discuss this with Apple accessibility.

We don't have any contacts there so we'd appreciate any help.

It's worth noting that visually hiding the MathML and hiding the HTML-CSS for accessibility would certainly work for both NVDA and VoiceOver right now.

Right. We're open to any solution that helps AT users the most.

[pkra]

[Ah, I somehow triggered a posting. Moving the second comment into the right place]

[NSoiffer]

I came across this Paciello Group post that includes tests on browsers and AT (updated Nov 2013). It recommends the off screen technique:

Use an off screen technique to remove content from visible display, but still available to screen reader users: Example code: .offscreen {position:absolute; left:-10000px; top:auto; width:1px; height:1px; overflow:hidden;}

<div class="offscreen">This text is hidden.

[sinabahram]

Please see this page for a few enhancements to that technique:

http://snook.ca/archives/html_and_css/hiding-content-for-accessibility

[fleizach]

Hiding MathML in an attribute would make touch exploration of math elements impossible with VoiceOver on iOS.... ditto for moving it off screen

The best option is to use MathML natively. Then it's there visually, it can be explored by touch and the native rendering semantics that are deployed when rendering in web engines is able to be leveraged by accessibility code to output relevant information.

[fred-wang]

I agree with Neil and Chris here, putting the <math> tag directly in the source code and hiding it (like what KaTeX and Wikipedia do or, alternatively, using offscreen technique) is the best option: then any accessibility tool can access the MathML source "naturally", without having to introduce some hacks to make it work specifically with MathJax.

The data-mathml option sounds horrible to me, so I definitely hope you won't go that way since it won't work with native accessible tree APIs as used by WebKit+VoiceOver and that is intended to be implemented in Gecko/VoiceOver, Gecko/Orca and VoiceOver/Orca (and any future combinations of web engine / AT that implements this kind of API).

If I understand correctly the <object> option, it is putting the MathML in a separate document, embedded as an object. My guess is that it will work better than the data-mathml option, but will still not be "natural" since the AT will will have to tell the user that he is entering/exiting an <object>. @fleizach probably knows better what will happen here, and any potential problems.

Someone commented that Wikipedia's CSS hidden technique has issues (there is indeed a bug entry about touch screen). As indicated by Chris, this obviously breaks visual rendering and the related a11y features (highlighting, zooming, exploring by touch...). But all the techniques discussed in this thread (including putting MathML in the shadow DOM) have this defect anyway (unless of course MathJax tries to do the job of ATs and reimplements features that should really be native). At least having MathML in the DOM, even hidden, will improve a11y support for non-visual output.

Finally, regarding the comments about greasemonkey scripts and extensions trying to extract data-mathml, this sounds an additional workaround over the data-mathml hack. I recommend to people to use extensions to automatically switch to native MathML instead, which will be much more efficient:

https://addons.mozilla.org/en-US/firefox/addon/mathjax-native-mathml/ https://github.com/fred-wang/mathjax-native-mathml-safari https://github.com/fred-wang/mathjax-native-mathml-greasemonkey

As a bonus, this will also make visual a11y features work.

[jcsteh]

I'll stress this again: visually hiding native MathML means an AT can't get the location information for the MathML. Among other things, that means no MathJax context menu access. As has already been pointed out, it also means no touch screen access. There are hacks to work around this (such as ATs using the role="math" parent for location information of the entire expression), but these are still AT specific hacks. The point is that this utopian idea that visually hiding MathML is nice and clean and doesn't require AT hacks is completely false. Until we're dealing only with native browser rendering of MathML, this is going to be hacky.

[fred-wang]

@jcsteh What do you mean by "hiding" and which comment are you referring by "hiding MathML is nice and clean and doesn't require AT hacks"? AFAIK, people working on MathML a11y confirmed that the CSS clipping hack used by Wikipedia and Kahn Academy works for non-visual a11y features (in particular Neil confirmed that it works with NVDA+MathPlayer). Are you suggesting to introduce a different hack for MathJax, which would move even farther away from native MathML? And what would be the rationale for that? (And as a separate question: which feature of the MathJax menu are you willing to expose?)

[jcsteh]

@fred-wang: You stated:

then any accessibility tool can access the MathML source "naturally", without having to introduce some hacks to make it work specifically with MathJax

That's true if you don't care about being able to locate the math on screen and you don't care about doing anything visual. My point is that if you do care about either of these things, you will need hacks, so visually hidden MathML actually doesn't eliminate the need for hacks at all.

As for the context menu, we don't plan to expose a specific feature, but there are certainly useful options in that menu. If we take the visually hidden MathML approach, there will be no way for a screen reader user to right click the math to access the MathJax context menu, since the MathML that the screen reader sees is off-screen and therefore you can't route the mouse to it. Note that data-mathml does not have this problem (though it's certainly pure evil), since the location information for the root MathJax node will be correct.

Visually hidden MathML seems to be the general consensus and that's fine. However, it will definitely prevent context menu access for screen reader users unless we want to implement another kind of horrible hack, and since you don't like data-mathml because it's a horrible hack, you're unlikely to like the kind of horrible hack needed here either.

[fred-wang]

That's true if you don't care about being able to locate the math on screen and you don't care about doing anything visual. My point is that if you do care about either of these things, you will need hacks, so visually hidden MathML actually doesn't eliminate the need for hacks at all.

OK, thanks for clarifying. I thought my comment was clear that I meant making non-visual things cross-compatible and that I'm suggesting the add-ons forcing native MathML for more advanced features. As I understand the choice of data-mathml will bring more visual things for NVDA (modulo additional adjustments) but will still not be useful at all (even for non-visual things) for other tools like VoiceOver or Orca. Personally, I'd prefer that the default is to have a minimal cross-compatible accessibility than more hacks for one specific AT.

As for the context menu, we don't plan to expose a specific feature, but there are certainly useful options in that menu.

I think the most important for an AT is to be able to read the mathematical equation, while the features in the MathJax menu are of less importance. Using the extension I mention, you can force native MathML while also keeping the MathJax menu. Alternatively, other add-ons like https://addons.mozilla.org/en-US/firefox/addon/mathml-copy/, https://addons.mozilla.org/en-US/firefox/addon/mathml-zoom/ or https://addons.mozilla.org/en-US/firefox/addon/mathml-font-settings/ expose equivalent features via the native Firefox contextual menu (which should be accessible!).

[jcsteh]

As I understand the choice of data-mathml will bring more visual things for NVDA (modulo additional adjustments) but will still not be useful at all (even for non-visual things) for other tools like VoiceOver or Orca. Personally, I'd prefer that the default is to have a minimal cross-compatible accessibility than more hacks for one specific AT.

In principle, I agree. However, it's worth noting that when I proposed this, I was aiming for open discussion and consensus with maximum benefit for all ATs, not just trying to get an NVDA specific hack into MathJax. I had no idea whether Orca and VoiceOver would refuse data-mathml, despite the potential advantages. I also wanted to make this as non-invasive as possible; I wasn't sure MathJax would be willing to drastically change their DOM structure for accessibility. Now that this is all very much clear, of course it makes sense to just go with native, visually hidden MathML and people will have to accept that screen reader users will never be able to access the MathJax context menu or do other visual stuff. At the end of the day, even this solution is a nasty hack--any visual hiding like this is a nasty hack--but so is non-native MathML rendering. :)

[fred-wang]

people will have to accept that screen reader users will never be able to access the MathJax context menu or do other visual stuff

Well rather than 'never' I'd say until MathJax uses native MathML as the default or until these people install an addon to force native MathML... In my opinion it's not at all a bad idea to focus on a standard approach (MathML in the DOM) for ATs, math content generators, and Web rendering engines and teach people how to use it.

A separate note on the MathJax contextual menu: as I remember MathJax makes this menu accessible by attaching event on each math equation despite the fact that the menu is not equation-specific (please correct if I'm wrong). If instead the menu was made globally available and not available for each math expression, we won't have this "hiding" issue.

[fred-wang]

please correct if I'm wrong

correcting myself: at least the copy & paste feature is equation specific.

[pkra]

RE @fleizach

Thanks for your comment, Chris! We really appreciate getting feedback from an Apple/VoiceOver point of view.

Hiding MathML in an attribute would make touch exploration of math elements impossible with VoiceOver on iOS.... ditto for moving it off screen

Yes, we are (painfully) aware of this.

The best option is to use MathML. Then it's there visually, it can be explored by touch and the native rendering semantics that are deployed when rendering in web engines is able to be leveraged by accessibility code to output relevant information.

I think everyone agrees that native MathML would be the ideal solution.

The trouble is that it's difficult to argue it would be "the best option" right now. The main reason is that

Then it's there visually,

is not actually the case.

IE and Chrome have no MathML support planned and WebKit's MathML support is not ready for professional production. (If there's interest from Apple, I'd be happy to put together a list of organizations that feel that way, e.g. large and small publishers in K-12 and/or research-level STEM, also sites like Wikipedia, StackExchange etc). Unfortunately, there is also no significant MathML development on WebKit right now that would indicate improvements might eventually arrive.

We're not judging. It's just the reason why we want (but really: we need) to figure out alternative means of making our rendering (and hopefully that from other polyfills) as accessible as possible. We would greatly appreciate any additional input.

For example, how should an SVG diagram be made accessible to a (VoiceOver) user? How should an author of complex typography (such as these examples) provide information for (VoiceOver) users?

Again, we really appreciate your comments, Chris. We hope, thanks to all the feedback we received here and elsewhere, we can provide a decent compromise for those users who need both a visually correct and AT friendly rendering.

---

RE @jcsteh, @fred-wang thanks for the additional comments.

[fred-wang]

I guess this is obvious, but it has not been mention explicitly: a CSS hack to hide the MathML can be reverted via a user stylesheet (which can be applied or unapplied at any moment). In practice, this is much easier to perform, more reliable, less subject to bugs and more efficient than a script synchronizing with the page's javascript and extracting the MathML for each formula.

[fleizach]

On May 1, 2015, at 8:12 AM, Peter Krautzberger notifications@github.com wrote: [...] IE and Chrome have no MathML support planned

This is a real shame in my opinion. It is a direct affront to users who need AT to read math equations.

and WebKit's MathML support is not ready for professional production. (If there's interest from Apple, I'd be happy to put together a list of organizations that feel that way, e.g. large and small publishers in K-12 and/or research-level STEM, also sites like Wikipedia, StackExchange etc). Unfortunately, there is also no significant MathML development on WebKit right now that would indicate improvements might eventually arrive.

I would like a list of bugs or issues that would bring MathML into production level quality so we can start working on those.

For example, how should an SVG diagram be made accessible to a (VoiceOver) user? How should an author of complex typography (such as these examples http://www.smashingmagazine.com/2013/02/26/creating-visual-hierarchies-typography/) provide information for (VoiceOver) users?

SVG elements can be made accessible with aria-label. In general this should work, but there are probably caveats in there.

As to the overall solution of exposing math content but hiding it -- I also don’t have any novel ideas.

<math display:none aria-hidden=false>

could provide some benefit.

It would be nice to be able to have some way to render the MathML if a screen reader is running. I know no spec allows for this at the time being.

So I would probably rather see the content be clipped, but still rendered somewhere so that positioning information would be available

Thanks

[sinabahram]

This won’t work:

<math display:none aria-hidden=false>

I believe you obviously mean:

<math style=”display:none” aria-hidden=false>

But, that’s not why it won’t work. it won’t work because display:none removes from the screen reader as well, regardless of aria-hidden.

[jcsteh]

Well, there is talk of having aria-hidden="false" override CSS display: none, but I don't think that's in a final spec yet. Certainly, it's not implemented for Windows browsers yet.

[fleizach]

FYI this behavior is implemented in WebKit

[NSoiffer]

If it's not displayed, then something else is likely taking up the space into which it would be displayed. Even if the specs were changed, why would this be better for location-based events than if it were displayed offscreen?

Neil

[fleizach]

It wouldn't be much better, hence my advocacy for just using mathml as it's intended to be.

[sinabahram]

Let me put it this way. This behavior is non-standard for over 80 to 90% of blind/low vision web users.

[pkra]

Thanks for the recent comments.

@fleizach since the display:none with aria-hidden=false only works on Safari, it's not an option for us since our goal is browser-independent output.

We had a call with a number of AT vendors and accessibility experts a few weeks ago to talk about possible solutions. This didn't get us to any specific solution but certainly a few steps closer -- thanks to everyone who took the time!

Right now we are considering the following.

• an extension that generates the internal MathML, embeds it hidden in the DOM as well as in a data-attribute (with matching ids etc).

We expect a performance hit for this extension. This might make it prohibitive (at least for now) to load this extension by default. For that case and for a first iteration we are considering:

• A user opt-in via the MathJax menu to load that extension even when the page author has not configured MathJax to load it. (Please note that work on this issue will happen in sync with #939 and other related issues).
• A discoverable opt-in for keyboard users. For example, via an invisible button that is revealed upon navigating to the first equation or using other visually hidden means to provide the information.
• Any other clever way of opting in, in particular we're still looking for an idea that would help sighted AT users. We'd appreciate any input on this.
• Finally, we will also consider loading the extension as a "post-processor" (i.e., after MathJax's visual rendering is done).
• These opt-ins would work on a per-domain basis using the usual MathJax cookie to store preferences.

Thanks again for all your input and any further feedback you might have.

[fsrg]

Would it be possible for an AT to set a flag in the MathJax cookie so that the AT's users wouldn't have to opt-in for accessibility?

[sinabahram]

This is not a good practice from an accessibility point of view. There’s no way we can convince all AT to do it, and while it’s appealing from a development standpoint, it’s not good to force users to opt into basic access.

[fsrg]

I don’t hold with your assertion that “this is not a good practice from an accessibility point of view.” We don’t need to convince all AT to do it. But those who want to make it easier for their users should have the option. And an AT user by definition wants basic access. And like all users of any type of software, they want it to be as hassle free as possible.

[sinabahram]

The last thing we need is to have an AT user jump through more hoops. We don’t ask users if they wat the alt text on an image or if they would like certain accessibility features turned on in a page. I really do understand the simplistic appeal to ask users, but I view it as an utter last resort. We need the math to “just work”, not have someone turn something on by finding it, and then having to reload a page and then having that setting lost in incognito/privacy mode, etc.

I strongly disagree with this approach.

[fsrg]

Sina, it’s possible you misunderstood my comment, assuming that you were indeed responding to my comment. I believe we are in agreement. I, too, want MathJax to “just work” with ATs. However, I think Peter ‘s objection to having MathML embedded all the time is that there is a performance penalty for the vast majority of users who don’t use AT with MathJax. Therefore, to try to come to some solution that would accommodate both sets of users, ATs and non-ATs alike, I suggested using the MathJax cookie as a way for the AT itself to do the work programmatically that the user would have to do manually.

Thanks, RG

[sinabahram]

First, sorry for lack of threading. I’m using github’s email response mechanism, and I don’t have any feedback about whether it groups or threads messages correctly. I am responding to the cookie message on this thread. Sorry for any confusion on that.

I hear you on the AT actuating the right values within the cookie, but there are many instances where I’m not sure that’s going to work. I feel that we still have a strong requirement for the user to indicate necessity of MathML when the AT doesn’t work or in a variety of instances where that value is lost. There is also a significant privacy issue here. what we’re proposing is that folks have to indicate the status of their disability or at least that they are using AT. And, for most users, they won’t even know this is going on. to me, that’s a pretty big deal, not because I care, personally, if a website knows that I’m blind, but because I know many people will and frankly, have perfect reason for doing so.

RE performance hit: Am I correct in my understanding that rendering the MathML off screen also incurs this performance penalty? If it doesn’t, Neil was telling me that there’s an example, and we need to find the URL of this example, that as long as the div is on screen, then the math can be positioned off screen, and then the user can still touch the visual math image but hear the MathML. We should verify that to at least address the off screen concerns from earlier.

My basic intended use case is that when a kid in school has a screen reader on and goes to a Wikipedia or other page containing math, it just works for them. I still think this is possible.

[fsrg]

@Sina “My basic intended use case is that when a kid in school has a screen reader on and goes to a Wikipedia or other page containing math, it just works for them. I still think this is possible.”

We agree on that: when running a screen reader with MathJax, the experience should be seamless.

[sinabahram]

Per the below/above

Neil and I verified that one can swipe to MathML that has been minimized to 1px by 1px, but not touch it. this seems perfectly acceptable.

[fred-wang]

Neil and I verified that one can swipe to MathML that has been minimized to 1px by 1px, but not touch it. this seems perfectly acceptable.

What about the MathJax menu btw? James said that you can not route the mouse to some "hidden" math, but for the special technique of 1px by 1px (which is not entirely hidden) it might be possible to do so.

what we’re proposing is that folks have to indicate the status of their disability or at least that they are using AT

Note that the Firefox native MathML add-on creates the cookie at page opening and delete it once the page is loaded:

https://github.com/fred-wang/Mathzilla/blob/master/mathjax-native-mathml/data/menu-cookie.js

and it really just says "I want to use native MathML" not that you are blind.

Of course it would be better if MathJax could just always put standard markup by default as KaTeX and Wikipedia do, with a CSS hack that could easily be reverted with a user stylesheet. But reading Peter's comment, the MathJax team is unlikely to accept this approach. All the approaches proposed would 1) require some work for users to switch to a mode with MathML and 2) make MathJax's performance even worse that it is already.