Phonetic Mode: Option to phonetically spell 1-letter words in Math blocks

[bhavyashah]

Mathematicians seem to enjoy notation overload. In particular, I encounter many Math problems where letters that sound far too similar are used together. I find myself needing to navigate Math blocks frequently just to tell the m's and n's apart, or the b's and d's, or the s's and f's. But even once in, commands like NVDA+. (laptop layout) to read the current character don't work. That bug may merit a ticket of its own. For this one though, I share that as motivation for a phonetic mode feature request. When enabled, MathCAT would read the phonetic corresponding to a letter instead of the letter itself for all 1-letter words in Math blocks. This would contribute considerable verbosity so it should be turned off by default. However, I think enable phonetic mode, read current line, and then disable phonetic mode would be a much better workflow than trying to navigate the Math block and arrowing around to try to hear the confusing letter in isolation a couple times.

[andrew-l-d]

I raised a feature request with NVDA on this issue in February - Add delayed description of characters in equations (https://github.com/nvaccess/nvda/issues/14661). Judging from comments, it is not an easy fix within NVDA.

Andrew

From: bhavyashah @.> Sent: Monday, June 5, 2023 5:40 PM To: NSoiffer/MathCAT @.> Cc: Subscribed @.***> Subject: [NSoiffer/MathCAT] Phonetic Mode: Option to phonetically spell 1-letter words in Math blocks (Issue NSoiffer/MathCATForPython#48)

Mathematicians seem to enjoy notation overload. In particular, I encounter many Math problems where letters that sound far too similar are used together. I find myself needing to navigate Math blocks frequently just to tell the m's and n's apart, or the b's and d's, or the s's and f's. But even once in, commands like NVDA+. (laptop layout) to read the current character don't work. That bug may merit a ticket of its own. For this one though, I share that as motivation for a phonetic mode feature request. When enabled, MathCAT would read the phonetic corresponding to a letter instead of the letter itself for all 1-letter words in Math blocks. This would contribute considerable verbosity so it should be turned off by default. However, I think enable phonetic mode, read current line, and then disable phonetic mode would be a much better workflow than trying to navigate the Math block and arrowing around to try to hear the confusing letter in isolation a couple times.

— Reply to this email directly, view it on GitHub https://github.com/NSoiffer/MathCATForPython/issues/48 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ADZAYAMPX4JJZQDYXZYH7T3XJWEOFANCNFSM6AAAAAAY2SJDIU . You are receiving this because you are subscribed to this thread.Message ID: @.***>

[NSoiffer]

I'm not clear what you want.

Do you want something "log" to spell itself "l o g" when reading? Or are you asking for letters themselves to somehow be spoken differently? If so, how would you want (for example) "m" to be pronounced? Use the NATO phonetic alphabet so that "m" becomes "mike"? Something else?

[andrew-l-d]

Neil, what I would like is the ability to expand to the military alphabet, but do not know if that is possible. It is possible with screen readers when reading in most applications, but does not currently work in MathType or MathCAT.

Andrew

[NSoiffer]

@andrew-l-d: Do you mean something different from referenced NATO phonetic alphabet the when you write "military alphabet"?

It would not be hard to add that to MathCAT. The question is how to control that speech. The obvious thing is to make it an option that can be changed in the preferences panel, but I suspect turning it on and off that way would be annoying. Do you have any suggestions? Maybe make it time based or character count based (both are harder). That sort of option turns itself back off after the specified limit is reached. Maybe there is a more intuitive auto-off feature so you don't need to keep changing the option.

Another option would be a keystroke to toggle the mode. It's hard to see how this feature is common enough to use up a (valuable) key combination though.

[andrew-l-d]

I had not heard it called "referenced NATO phonetic alphabet " previously - take "military alphabet" as synonymous. Most screen readers will present the expanded form when pressing the key to present the character under the cursor twice. NVDA has recently introduced a feature which can be toggled on/off under Screen in Settings which works as follows. When arrowing right or left, if pausing for a second the expanded form is announced. I really like that approach, as it is inobtrusive most of the time. If either option could be implemented in MathCAT, that would be great. I agree that having to toggle in Preferences would be a bit tedious, but certainly better than not having the option if it cannot be implemented otherwise. Andrew

[NSoiffer]

That sounds like it might be doable and MathCAT could use the NVDA setting (as it does for how caps are spoken).

I'll transfer the issue to the python repo because that is where the NVDA implementation lives and that is where this feature (with this suggestion) would get implemented. I'll see if I can mimic the code that NVDA uses (hopefully I can find it... and maybe even use it).

[bhavyashah]

I think I originally made this ticket when I still wasn't using/aware of the relative speech rate. With that feature though, I think I'd be better served by the ability to use NVDA review cursor commands in Math blocks, i.e. pressing NVDA+. twice (laptop layout) to phonetically report the current character. This is because a reduced Math speech rate is enabling me to process a majority of Math content, meaning that I think I now feel like I'd prefer to check/confirm my understanding with phonetics in specific cases rather than have them auto-reported for entire equations. Either way, the two are not mutually exclusive. Please let me know if discussion on that fits within this ticket or if I should open a new one. P.S. I have just resumed engaging with Math content after a summer break, so some thoughts may be fluid.

[bhavyashah]

Another change of heart: pressing Space twice itself could read the current expression phonetically.

[NSoiffer]

Helping to resolve sound-alike characters sounds like a good idea. If NVDA has a feature that (sort of) does what is wanted, I think the best thing is for MathCAT to:

1. use the same keystroke/menu setting/??? to invoke the feature
2. try to make MathCAT replicate as much as possible the feature

From the discussion so far, it sounds like @andrew-l-d likes the Settings:Speech:Delay descriptions for characters on cursor movement feature. @bhavyashah, would that solve the problem for you if I added that to MathCAT?

If I do add some feature, should it only run when navigating at a character level: either NavigationMode=Character or zoomed in to the character level. (Still not fully implemented ). If it should work at a higher level (e.g., "x plus 1" or "x squared"), what should it do in those cases? Spell out the entire phrase using the military/NATO alphabet?

[NSoiffer]

Note to self: if the NVDA feature above is the one to mimics, see

• characterProcessing.py: getCharacterDescription(locale: str, character: str) -> Optional[List[str]]
• called by: speech/speech.py: getCharDescListFromText(text,locale)

I didn't see where the timing delay was coming from.

[andrew-l-d]

Neil asked, "If I do add some feature, should it only run when navigating at a character level: ...?" My suggestion would be yes.

[bhavyashah]

Personally, I do not use the Delay descriptions for characters on cursor movement feature in NVDA. It is verbose when I don't need it (which is most times), requires me to wait, and only works for characters. I prefer to use the NVDA review cursor commands (read current character command pressed twice, read current word command pressed thrice, etc.) FOr MathCAT, my ideal implementation would be extending the read current position command such that pressing it twice phonetically reads the Math at the current position. Notably, this would work at any level, not just character level. Also, it would not try to convert things like exp/exponential, numbers, symbols +-/, etc. into phonetics. Basically, I don't want to need to zoom in to the lowest level before I can get the phonetic for a letter variable or a group of them (s, m^2n) and then work my way back up to the level I actually want to parse the expression at.

[andrew-l-d]

"It is verbose when I don't need it (which is most times), requires me to wait, and only works for characters." But if you do not need the phonetic spelling you do not have to wait for it, so no time lost. I agree though that double pressing to get the phonetic spelling works well. However, I am very doubtful that it could be made to work in MathCAT because it works with the review cursor and the review cursor does not access the MathCAT content.

Andrew

[bhavyashah]

The worry is that the verbosity of delayed character descriptions will interfere with my ability to cognitively process what I've just reviewed (esp. when reading a higher-level Math textbook or when working on a dense homework problem). This may lead to time loss indirectly by needing me to re-read the content to clear my head of the phonetic I just extraneously heard. Again, this is a small concern and I'd gladly have delayed character descriptions at character level over the status quo.

-- Kind Regards, Bhavya Shah B.S. in Mathematical and Computational Science | Stanford '24 LinkedIn: https://www.linkedin.com/in/bhavyashah125/

[andrew-l-d]

I would strongly suggest that, if introduced, it would only work at character level. The way it works in NVDA outside MathCAT is that the user chooses whether or not to enable it. That would be the ideal if it can be implemented in MathCAT. On the other hand, if a double-press could be implemented to produce the phonetic spelling, that would a very good solution. But, given that the review cursor does not do its thing within MathCAT, that may not be possible.

Andrew

[bhavyashah]

I think we agree that:

• Pressing Space twice (or re-assigning Ctrl+Space or Ctrl+Shift+Space or pick another favourite key binding) phonetically spelling letters at any level is the ideal solution (absent any consideration of implementation complexity)
• Supporting delayed character descriptions when the user has zoomed in all the way to just the letter is the second best implementation Please let me know if I misunderstand your view.