search

edrlab / thorium-reader

A cross platform desktop reading app, based on the Readium Desktop toolkit
https://www.edrlab.org/software/thorium-reader/
BSD 3-Clause "New" or "Revised" License
1.75k stars 153 forks source link

MathML expressions in EPUBs are not readable with NVDA in some installations #1709

Open ways2read opened 2 years ago

ways2read commented 2 years ago

On Windows 11 computer start NVDA 2022.1

Launch Thorium Reader 2.0.0 and open the test EPUB Accessibility Tests Mathematics

When encountering math expressions you just hear 'clickable' with either Access8Math version 3.3 or MathCAT 0.1.14 .

Go to the web page Quadratic formula page in Wikipedia and the math is readable and navigable with Access8Math and MathCAT as expected.

Reading math with this combination worked briefly on this computer before it stopped. I have uninstalled Thorium, NVDA, MathCAT and reinstalled but still broken. Math in Thorium with NVDA works for some people and not for others.

ways2read commented 2 years ago

Same machine, uninstall Thorium Reader (store version) and install version 1.7.3. Math is now read with MatcCAT (in all cases MathJax is turned on so the math is displayed visually).

NSoiffer commented 2 years ago

In some experiments today when math wasn't being read, NVDA reports "not math" when trying to navigate the math. This means NVDA didn't see this as math, which very likely means the hidden MathML is not there. In looking through the code, I see in main\streamer\streamerCommon.ts the line:

    // enableAssistiveMml: true,

enableAssistiveMml is what controls whether the hidden MathML is present or not. I know that in most MathJax configurations, this is enabled, but maybe somehow MathJax is getting into a state where it doesn't get enabled.

Looking through the history in this file, this line was introduced in https://github.com/edrlab/thorium-reader/commit/910c3c59133d41926ee086c807029fbee5a7cac3. When introduced, it was commented out. Is there some reason it isn't set to true?

ways2read commented 2 years ago

I wondered if the screen reader was not being detected in some cases and MathJax was generating its own accessible MathML description (this messed up screen readers in #1602). However, that doesn't seem to be the case:

danielweck commented 2 years ago

Thank you Richard for this very helpful analysis (as discussed in private emails).

Neil, we tried various different tweaks in Thorium's MathJax configuration, but clearly we are still experiencing issues with some combinations of screen readers + optional math plugins. We also enable MathJax's dynamic generation of accessible text (speech utterance) which is helpful in cases where there is no authored alttext (ref. "speech rule engine" by @zorkow ).

Side note: this caught my attention in the MathJax issue tracker:

NSoiffer commented 2 years ago

The MathJax PR doesn't seem relevant. First off, it is for "legacy v2". Thorium uses V3 I hope. Second it goes from what I guess is a bad value to "false", so it turns it off. It does appear that V3 has the setting as true as the default, so I doubt my suggestion solves the problem, but I don't know why it would hurt.

On Tue, Jun 21, 2022 at 3:27 AM Daniel Weck @.***> wrote:

Thank you Richard for this very helpful analysis (as discussed in private emails).

Neil, we tried various different tweaks in Thorium's MathJax configuration, but clearly we are still experiencing issues with some combinations of screen readers + optional math plugins. We also enable MathJax's dynamic generation of accessible text (speech utterance) which is helpful in cases where there is no authored alttext (ref. "speech rule engine" by @zorkow https://github.com/zorkow ).

Side note: this caught my attention in the MathJax issue tracker:

https://github.com/mathjax/MathJax/pull/2463/files

— Reply to this email directly, view it on GitHub https://github.com/edrlab/thorium-reader/issues/1709#issuecomment-1161559891, or unsubscribe https://github.com/notifications/unsubscribe-auth/AALZM3ARNUXGC4RBFV6BSXDVQGKJDANCNFSM5YK5NRWQ . You are receiving this because you commented.Message ID: @.***>

NSoiffer commented 2 years ago

@danielweck: is there a way to view the DOM in thorium to verify whether the MathML is there or not?

danielweck commented 2 years ago

yes, debugging tools can be accessed with the automated builds downloaded from the GitHub releases, or with local dev builds. in other words: production builds / official releases prevent access to the Chromium Web Inspector, Dev builds don't.