Screen reader support

[asavoy]

The aim is to improve accessibility of MathQuill to vision impaired users to meet the required guidelines.

Accessibility guidelines:

• TODO

Milestones

• [ ] Milestone 1. Basic support for MathQuill's read-only mode
  • We assume user familiarity with latex syntax
  • Hence it is sufficient to read out the latex value
• [ ] Milestone 2. Basic support for MathQuill's editable mode
  • Again, assume user familiarity with latex syntax
  • Hence the editor UI should revert to a simple text input field where they may input the raw latex
  • How do we explain which latex commands are available? Especially the tricky multi-part ones like \sqrt[3]{x} and \int^1_0.
  • How should we invalidate bad latex input?
• [ ] Milestone 3. Enhanced support?
  • Support nested editables: \editable{...}
  • How would we guide users to using the right latex commands?
  • Better guidance to reading/writing latex?
  • Relaxing dependency on latex syntax?

Software requirements:

• Must support 2 latest versions of JAWS, 16 and 17
• Mac/iOS VoiceOver support is a "nice-to-have"

From: https://trello.com/c/XiyOJaHP/8808-mathquill-screen-reader-support

[laughinghan]

Re: static math (read-only mode)

I don't think it's going to be sufficient to read out the LaTeX value. Anything other than flat arithmetic text and ^2 is quite incomprehensible when read out, like reading out a sentence letter-by-letter.

There's good news: there appears to be surprisingly good support for reading out MathML by screen readers including both that you specified, according to MathJax, and the math is read out reasonably comprehensibly.

MathQuill's read-only mode could use essentially the same strategy as MathJax's AssistiveMML extension, KaTeX, and Wikipedia, which is to have hidden MathML (<math>) alongside the HTML that visually renders the math:

(MathJax doesn't do this by default to avoid a "performance hit", which I think is preposterous seeing as KaTeX does this by default and is orders of magnitude faster than MathJax, and also significantly faster than MathQuill for that matter.)

There is another, more unorthodox possibility which is to skip the MathML step and just generate and hide the text we'd like to be read ("begin fraction 1 over 2 square root of x end fraction"). Implementing Nemeth "mathspeak" (from mathquill/mathquill#9) would be straightforward; also, ChromeVox's math speech rule engine is open source.

Re: math fields (editable mode)

I think using LaTeX syntax here would be even worse than in read-only mode. One of the main usability problems with computer languages is strict syntax/the existence of syntax errors, which is exactly why WYSIWYG editors are important: MathQuill makes it impossible to write syntactically invalid LaTeX. (It's still however possible to write nonsense; how were you planning on complaining if they write 3x^=? You could use the same system to complain about bad LaTeX, I guess.)

But, there's good news here, too: there's an attribute called aria-live that can be used to get screen readers to announce things, with decent user agent support, which could be used to announce the edit you just made to the math ("inserted fraction" or something, I dunno). With enough thought and experimentation I think this could be a viable way forward.