Possible Use Case: Text Structure

[carmacleod]

This is similar-ish to Issue #4, which says that AT needs to be able to query focus, selection start & end, etc. in a canvas [or contenteditable] text editor.

AT also needs to be able to query which word is at a given position in the text editor. (Not sure if "character/sentence/paragraph/etc at position" needs to be queried also, but perhaps).

This use case is best described by a recent email conversation I had with a screen reader user regarding our Code Editor widget...

Me:

I don't like the way navigating by words with ctrl+right arrow works with a screen reader, because the browser tells the screen reader where the word boundaries are, and browsers don't expect all of the special symbols that are found in code. So things like brackets, dots, equals signs, etc, cause the screen reader to repeat some words/symbols and skip others. Here's a demo: http://libingw.github.io/OrionCodeEdit/ It works better in Firefox than in Chrome. I tried it with both NVDA and JAWS (configured to speak all punctuation). Type up/down arrow to hear complete lines of code read correctly, and then type ctrl+right/left arrow to move by words. (Note that you can type ctrl+M to change the tab key mode from "insert tabs" to "navigate to next tab stop" in order to get out of the editor so that it is not a keyboard trap). Is this odd-sounding word navigation something screen reader users would expect in a web-based code editor? Do you think they would just not use it? (Since line-based navigation, typing characters, moving by character, etc work well).

Screen reader user's reply:

This is an interesting problem, that impacts so much more than code editing. You are correct, in browsers word by word navigation does not read well. For that matter, it is horrible and frustrating. This is partly why it is so difficult to just say that we will move document or spreadsheet editing into the cloud on a web based platform. Theoretically it is possible, but it is not productive. I avoid word by word navigation as much as I can, it is not worth it. I am a JAWS user, and I also have the fortune of owning a braille display. I use my braille display to position my cursor anywhere I need on a given line, and for working with code, I would say it is mandatory. I just can't imagine working with code in any other way, especially when it comes to languages where spacing matters.

Here's what's happening in a bit more detail. Assume Firefox and IAccessible2, for example. When the user types Ctrl+right arrow, our Code Editor moves the caret forward by one variable or symbol, depending on the line of code. The ctrl+right arrow is passed through to the AT, which sends IAccessibleText::textAtOffset [1] with the caret offset and a boundary type of "word" [2] to find out what the new word in front of the caret is. Then the browser answers for us what it thinks the word is, but it is wrong because it doesn't realize that if, for example, the caret is between x and = in x= then the answer should just be = and not x=. Somehow, the browser needs to inform us that the AT is asking for a word, so that we can answer what that (code) "word" is.

I'm not quite sure what the API for this would be if it was included in the AOM. I realize that this is "Defer for now" future work, but I just wanted to put it out there for consideration.

[1] IAccessibleText::textAtOffset(offset, boundaryType) -> startOffset, endOffset, text http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/interface_i_accessible_text.html#a9bd84b099ac7ce2435280549e90c8827 [2] IA2_TEXT_BOUNDARY_WORD http://accessibility.linuxfoundation.org/a11yspecs/ia2/docs/html/_accessible_text_8idl.html#ad3da05c9bed9f44b1e52eb4adb2924f6

[alice]

Thanks for writing this up! Definitely worth keeping in mind.

[carmacleod]

Thanks for reading it, and for keeping it in mind! :)