Implement Inline Source Renderer for Accessibility

I am a visually impaired computer science student close to graduation who is attempting to self-study mathematics, particularly linear algebra and differential equations. As it currently stands, MathJax is inaccessible to me due to the basically nonexistent support for MathML. The upshot of this is that all the pages written with MathJax are inaccessible. I am aware of my options including mathPlayer, and if necessary I will explain exactly why it fails to work for advanced mathematics. Regardless, MathJax can display source for me. Unfortunately, the process to get to this source must be done per-equation and involves at least 3 advanced functions of my screen reader and at least 10 keystrokes. I would like to be able to set MathJax to display source instead of rendered content inline so that I can actually make progress on such sites as this linear algebra textbook and Stack Exchange. My preferred format is LaTeX, and it is my understanding that it is a somewhat common skill among blind STEM majors to be able to at least read it (Wikipedia alt text/article source for example is a great resource for us). I believe that implementing a renderer that simply displays the source is not a major change; the time it takes to start showing up on actual web sites obviously might be. MathML might finally be going somewhere, but I don't expect to see good support for it for another year at least. Advanced mathematics will be the last to be made fully functional. This would make a very wide variety of things accessible that are not currently, opening all sorts of avenues for me and other people in my situation. It should work on all platforms, regardless of MathML support: I don't expect linux will have it for a while, for example, and some phones probably will never will. I would like to at least discuss doing this. I am open to attempting to implement this myself and probably have the programming skill required, but I am clearly unable to test any visual aspects that I might or might not break in the process. The above is clearly useful for accessibility, but may also be useful for debugging. I am not sure on this, but it might be useful to some for that alone. If this is the inappropriate venue for this discussion, I would appreciate being pointed in the right direction.

Austin,

As the developer for math accessibility in MathPlayer, I'm very interested in hearing why it doesn't work well for you and what you feel would improve its accessibility.

Since someone asked, I suppose this is no longer off topic: We've spoken before in the distant past. I am aware of the new one and tried to request a beta but never got a response from you guys: Jamie with NVDA wants me as a tester, because I'm one of the few people he has who can give feedback both as a programmer and someone who knows math. Since someone asked, I suppose this is no longer off topic: The old one is missing navigation, requires screen readers to support it specifically (none have to my knowledge, at least not to the point of braille), requires a major downgrade to internet explorer, was (for the bit that I played with it) extremely verbose, couldn't let me interact with matrices or tables, inserted a bunch of punctuation that prevented me from even using word navigation, and decided to spell out otherwise intelligible punctuation as multiple words. I then uninstalled it because, for example, Wikipedia alt text is more helpful and sane and generally more useful. On account that I can't actually get books in MathML anyway and need to be able to share math with others, I'm really starting to seriously learn LaTeX, and unless there is a major, epic, and earth-changing event coming up, the further you go in math the less helpful MathPlayer will be. Also, it is my (perhaps inaccurate) understanding that MathJax is using presentation MathML, not content mathML. Inline source might be better just for that reason, but it was an old article I saw it in. Looking at NVDA source, the new MathPlayer will only ever work on windows because you guys made it (1) a COM object and (2) closed source as far as I know. Unless you've got a second version somewhere, this change may be worth it just for all the other platforms that get left out. As a blind programmer, MathJax source is closer to what I am accustomed to anyway, and I'd make the argument that there's probably a lot of other people similar to me (I do not have data). I think that covers all my problems with it. On account that I believe this to be a small change, don't expect any new MathML technologies to have an impact at all for me for at least 6 months and probably closer to a year, and found out that it is recommended that mathJax be used via a central repository, I am still strongly convinced that this is a good idea. I need access now, and this can give it to me with only 10-15 or so keystrokes per web page per History clear (which is much better than 10-15 per equation, and the keystrokes/advanced screen reader functions required is an entirely separate issue).
Given the lack of interest, I am thinking that this may belong on the mailing list that I should probably go join at this point.

I fear this is not that relevant to the bug list. If you subscribe to Blindmath@nfbnet.org, that would probably be a better place to continue this.

The one MathJax-related part I can clarify is that MathJax internally uses something very close to presentation MathML. This allows for easier conversion to various output forms, including native MathML renders. If the input was TeX, as it is in your linear algebra example, getting that could be useful to you, but if the input is MathML, I guarantee you don't want to read that. Currently, MathJax is not capable of turning MathML into TeX.

Since you are a programmer, you might consider working on an output jax that displays TeX. That way, the only changes you would need to make would be to set the output jax to that jax, and you would only need to do that once/domain. At least for a lot of common math, converting MathML to TeX is not that complicated.

If you are going to continue on in math or computer science, I do encourage you to learn more TeX as your professors and likely your fellow students will be using it also, especially in grad school.

You raised a number of issues related to MathPlayer, so let me try to address them one by one:

The new version of MathPlayer that will be coming out soon is not tied to IE and supports navigation. It also supports braille, but since it is meant to be integrated with a screen reader that is also using the braille display, it is dependent upon the screen reader calling MathPlayer for the braille and it putting it on the display. If they don't access the braille functionality in MathPlayer, you should be talking to the folks that develop your screen reader. If that's NVDA, I'm sure they'd appreciate your help making it happen. The new version is currently Windows-only, but we expect to have a web-service version as part of a V2. Since you use NVDA, you must be a windows user, so I would think the new version would be useful to you. We haven't sent it out to beta testers yet -- there are still a few more things to do before we can do that.
MathPlayer 3 and this new version have a Windows control panel that allows you to adjust the verbosity. Did you try the "terse" setting and find it still too verbose. If so, I'd appreciate examples of what you found verbose and an alternative terse way of speaking it. Also, you can change the settings from "blindness" to "learning disabilities" and you'll remove the verbosity associated with disambiguation for 2D notations. If you have no usable sight, that might not be useful to you.
The new version allows you to interact with matrices and tables as part of the navigation. I hope that when you try it out, you let us know about any improvements you would like to see for table navigation.
The punctuation that MathPlayer 3 inserts is needed to add some prosody to the speech to make it more understandable when the screen reader doesn't inform MathPlayer of the speech engine being used. If it does let MathPlayer know about the speech engine, nicer sounding speech can be generated. Again, you need to talk to and/or work with your screen reader provider to make this happen. It is only a few lines of code.
Could you give some examples of higher math that you feel makes MathPlayer less useful? Currently, MathPlayer has specialized rules that can be turned on in the control panel for Geometry, Probability & Statistics, and for Calculus. We know more can be done, but have not received any specific requests for other areas to cover. If there is a particular field where you feel the notations could be better described, we'd be eager to work with you to include those descriptions.

I hope I covered everything. We realize that MathPlayer's speech can be improved. Improvements happen when users such as yourself let us know what problems you've encountered and what things you would like to see changed. So please contact support@dessci.com with bugs and suggestions.

@camlorn sorry for the delayed response, most of the MathJax team was "out of office" one way or another.

Thanks for the posting and your interest in contributing. This issue tracker is definitely the right place for a feature request. For general MathJax development discussions we have a Google Group at mathjax-dev@googlegroups.com.

Implementing an output showing TeX sources wouldn't be hard (see this code snippet) but I think it might not be the best approach. It doesn't really fit with MathJax's modular nature. But mostly because it seems unnecessary. I think it might be more useful (and even easier) to hide the MathJax output from assistive technology while providing TeX source when that's available (or asciimath input, and even useful annotation-xml from MathML input). In the long run, a MathML to speech text conversion would really be ideal; with ChromeVox there is an open source implementation around to learn from.

In any case, if you're interested let's move the MathJax part of this discussion to our dev list.

Regards, Peter.

I'm wondering if a Greasemonkey script that disabled MathJax so that it didn't run at all might not be what you really want. That way the origin TeX would remain in the page and there would be no delay in getting it to show up. The only issue, is that the TeX delimiters would also remain, and it may be that you want to have those removed.

I'm assuming, here, that you are only talking about pages that have TeX as the input, not MathML. we don't currently have a MathML-to-TeX translation (only the other way around), so there is no easy way to get the TeX from something that didn't start as TeX. That is one reason why a TeX output jax is not part of MathJax at the moment (since the output jax are supposed to be independent of the input formats).

It would be possible to have a Greasemonkey script that includes your own custom output jax (and that overrides the page's selection of output jax) that puts the original format back into the page as the output. I don't think it would be a very complicated one. But I doubt that we will want to include that in MathJax itself. As Peter mentioned, we are already investigating other approaches to handling assistive technologies that seem more promising in the long run.

I went missing because of finals week. The way that I've seen MathJax's show source makes it look like, from a user's perspective, you are already doing the conversion. Since this is not the case, this is perhaps not as easy as I first thought. If I need to resort to temporary solutions that kill MathJax, I will do so. I probably need to look at it, as I can't afford to wait a minimum of 6 months and a maximum of forever. A $-sign is not a big deal. As for MathPlayer, I believe the way things worked out with your first version is, no screen reader ever bothered, and Jamie's made it pretty clear to me he doesn't want to with the old one (but he already has some support for the new one, apparently). I couldn't get it working well enough to get past algebra: its inbuilt speech does not work at all well with default SAPI voices, the punctuation and use of nonliteral symbol MSAA labeling means that I cant even read it like I'd read code, and it really just failed for the stuff I need to look at. It took me an entire afternoon to get it working in the first place, and I really don't want to be running out of date Internet Explorer versions for a lot of reasons. My feedback is it fails completely. As an actual user, I'd be super happy with LaTeX in so many ways, but understand that I'm one of the few blind people who considers math a fun hobby and wants to self-teach abstract and linear algebra among other things, and that probably only my particular class of user would be satisfied with such. I still strongly believe that the new one should be an open source C library. If your web service is going to expect the screen reader to round-trip all the time, no one will be pleased, really: the latency on that will be noticed by anyone who actually goes at a respectable speech rate. I assume you know this though, and intend it to be something else. The point of open source here is cross-platformness and that people will always have access, even if you disappear; I doubt there's enough blind people for anyone to bother competing with you in any way. Regardless, I think both issues have moved past this ticket and onto separate mailing lists at this point. I'll need to go join them, I guess.

Leaving this open; it's not a bad feature request imho.

The way that I've seen MathJax's show source makes it look like, from a user's perspective, you are already doing the conversion. Since this is not the case, this is perhaps not as easy as I first thought.

MathJax can only show TeX-sources if either the input was TeX or if the MathML contains annotation-xml with TeX.

This was fixed in v2.6 via the new PlainSource output, https://github.com/mathjax/MathJax/pull/1329

mathjax / MathJax

Implement Inline Source Renderer for Accessibility #788