Screen reader not automatically following hyperlinks destinations into content webview (from TOC GUI, etc.)

[GeorgeKerscher]

In our Reading System working group, we discussed Thorium not always linking to the correct position in a chapter. Using the epubtest.org Basic Fundamental test book, Richard, Prashant, and George find the linking to be undependable.

We experience this behavior using the TOC and with go-to-page. Using both Jaws and NVDA, I open the table of contents and go down to the navigation tests. I press enter, and am taken to the correct file, but not to the exact location. I am taken to the top of the file. If I activate the hidden screen reader link "___" it will sometimes work and sometimes not.

I have unpacked the same title and directly use Chrome to open the nav doc, and the deep linking works fine.

We have marked this as pass in the epubtest.org results page, because sometimes it works. However, I think it best to go back and mark it as fail.

[danielweck]

In my tests with VoiceOver on the latest version of MacOS, when I navigate to an authored heading using Thorium's Table Of Contents GUI, visually the indicated keyboard focus seems to be correctly redirected into the embedded content webview, but the screen reader remains "at the top" of the frame instead of moving into it. I then press tab once to reach the special underscore link which then reliably takes me to the link destination (i.e. TOC heading, page number, bookmark, etc.). This indicates that the DOM keyboard focus also remained stuck "above" the main content frame (i.e. physical webview boundary which is technically isolated via shadow DOM and as an out-of-process separate renderer, for security and performance reasons primarily). Screen readers clearly have a hard time interpreting the nested Electron / Chromium webview (no surprise this works fine when opening the HTML document in a web browser, as there is no hard level of indirection / boundary to cross). Definitely something I would love to solve, but so far I haven't found any technique that automatically forces the screen reader focus deep into the webview. Thus why I added the special underscore link at the top of the document, to aid navigation when the screen reader fails to synchronise its own cursor.

[danielweck]

There is an existing Electron bug which documents focus issues with webview components (which are under the hood out-of-process iframes inside a shadow DOM): https://github.com/electron/electron/issues/31918

[danielweck]

Hello @nvdaes @GeorgeKerscher good news, I found a technique that works very reliably with VoiceOver on the latest version of MacOS. I didn't test with other configurations, could you please test the latest alpha which is currently building, see the status link to track progress (the Windows installer should be ready within the next half hour): https://github.com/edrlab/thorium-reader/actions/runs/1584412998

Fixed via https://github.com/edrlab/thorium-reader/commit/c86bc39bd8c153ff635daff745c6af29773ad524

[danielweck]

Also, I found that creating bookmarks at the precise document location currently read aloud by the screen reader (i.e. similar to a sighter user doing a mouse click on the text then invoking the bookmark shortcut) was very reliable in scrolling mode (i.e. not paginated mode, which by the way is automatically turned off when reading documents with MathML / MathJax). I managed this with VoiceOver and the CTRL OPTION SPACE keyboard combo, which seems to somehow activate the DOM element currently accessed by the screen reader (i.e. the AT's internal cursor, which is not necessarily in sync with the DOM "scrolled into view" element). This activation triggers Thorium's regular reading location update, which means that the reading system's internal state now considers this DOM element as the current reading location which is used when invoking the bookmarking feature (CTRL B). Now, on the downside, I noticed that sometimes the CTRL N etc. keyboard shortcuts stop working, I have to ALT TAB outside of Thorium and come back to it in order to restore them, or sometimes just changing the VoiceOver cursor by going up a level and coming deep again. This doesn't occur when the screen reader is turned off, so I guess there is some kind of interference. I look forward to receiving your NVDA and JAWS feedback (and Narrator?). I am unable to test at the moment due to the audio drivers being broken in my Windows virtual machines (Windows 10 and 11).

[GeorgeKerscher]

I have been testing with Thorium v1.7.4-alpha.1.1584412998

I have found the direct navigation from the TOC, in goto Page, and internal links to be highly accurate.

[nvdaes]

Hello Daniel and George, my results are also very accurate for pages, TOC links and search results. The special link labelled as underscore underscore (__) is not needed. About bookmarks, the underscore underscore link is not needed with the latest alpha, though the cursor doesn't always move to the precise position. The concern here is that, if it's moved after the position that we want to mark, blind users don't know where the previous text is placed. What about a configurable option so that bookmarks are placed in the previous object in the DOM? Otherwise, my recommended approach with NVDA is to use the placeMarkers add-on. Thanks

[danielweck]

Goerge, Noelia, Thank you for your reports.

Regarding bookmarks, just like any other preserved reading locations, search matches, and authored navigation links, the granularity is a DOM element, meaning that when the user clicks the mouse cursor in the middle of a very long paragraph of text without any DOM elements such as inline span or block div, Thorium records the parent element. Consequently, screen reader playback and the visual highlight can sometimes starts a long way before the actual pointed word / sentence. Now, it is important to note that the mere fact of vertically scrolling the rendered document (or turning pages horizontally in the paginated case) causes Thorium to choose the topmost DOM element in the visible content viewport as the so-called "reading location". By default that is what the bookmark "red ribbon" skeuomorphic metaphor implies, i.e. the reading location can be anywhere inside the displayed two page spread, or the visible area of the vertical scroll. It is only when the user interacts with the mouse in the document that Thorium can infer a more precise reading location. In my tests with Voice Over I was able to use CTRL OPTION SPACE to "select" / "activate" the currently-spoken utterance which the screen reader visually outlines with a thick black border. In essence, this would have been equivalent to me using mouse click inside that paragraph/sentence of text. My main issue during the Voice Over tests was that sometimes keyboard shortcuts like CTRL B (bookmark toggle, or bookmark add when selecting specific text in the document) or even CTRL N to open the navigation panel stopped reacting ... like if there was some sort of interference in the detection of key strokes. ALT-TAB usually cures this, or moving the screen reader cursor up and down in the nesting levels (for example coming out of the webview temporarily). Finally, I agree that the underscore special link might become obsolete for the most part, but I have experienced cases where the scroll position was temporarily lost so having this link to return back to the recorded reading location (e.g. TOC hyperlink destination) seems like a useful feature.

[nvdaes]

Thanks Daniel, with NVDA, I press NVDA+enter to activate the current object and the position of bookmark is very precise.

2021-12-16 12:31 GMT+01:00, Daniel Weck @.***>:

Goerge, Noelia, Thank you for your reports.

Regarding bookmarks, just like any other preserved reading locations, search matches, and authored navigation links, the granularity is a DOM element, meaning that when the user clicks the mouse cursor in the middle of a very long paragraph of text without any DOM elements such as inline span or block div, Thorium records the parent element. Consequently, screen reader playback and the visual highlight can sometimes starts a long way before the actual pointed word / sentence. Now, it is important to note that the mere fact of vertically scrolling the rendered document (or turning pages horizontally in the paginated case) causes Thorium to choose the topmost DOM element in the visible content viewport as the so-called "reading location". By default that is what the bookmark "red ribbon" skeuomorphic metaphor implies, i.e. the reading location can be anywhere inside the displayed two page spread, or the visible area of the vertical scroll. It is only when the user interacts with the mouse in the document that Thorium can infer a more precise reading location. In my tests with Voice Over I was able to use CTRL OPTION SPACE to "select" / "activate" the currently-spoken utterance which the screen reader visually outlines with a thick black border. In essence, this would have been equivalent to me using mouse click inside that paragraph/sentence of text. My main issue during the Voice Over tests was that sometimes keyboard shortcuts like CTRL B (bookmark toggle, or bookmark add when selecting specific text in the document) or even CTRL N to open the navigation panel stopped reacting ... like if there was some sort of interference in the detection of key strokes. ALT-TAB usually cures this, or moving the screen reader cursor up and down in the nesting levels (for example coming out of the webview temporarily). Finally, I agree that the underscore special link might become obsolete for the most part, but I have experienced cases where the scroll position was temporarily lost so having this link to return back to the recorded reading location (e.g. TOC hyperlink destination) seems like a useful feature.

-- You are receiving this because you were mentioned. Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-995707425

[nvdaes]

Hi daniel, thanks for your explanation. I'm activating objects in NVDA by pressing NVDA+enter at the cursor position, and now bookmarks are very precise, that is, if my cursor is iat the begining of a paragraph, then I press NVDA+enter, then control+b, control+n and then I press enter on the bookmark, the cursor goes to the specific paragraph.

[GeorgeKerscher]

Hello,

Using the NVDA key + enter that Noelia suggested works for me as well.

Daniel, can sighted users use the Annotation feature? I don’t see a hot key for annotations. I have also suggested a hot key for “where am I.” RedShelf have a hot key and it provides the heading previous to where the RS thinks it is at and the page number. The pop up window is accessible and is open for about 10 seconds and closes, and one can hit esc to close it as well.

Best

George

From: Noelia Ruiz Martínez @.> Sent: Thursday, December 16, 2021 5:03 AM To: edrlab/thorium-reader @.> Cc: George @.>; Mention @.> Subject: Re: [edrlab/thorium-reader] Screen reader not automatically following hyperlinks destinations into content webview (from TOC GUI, etc.) (Issue #1594)

Hi daniel, thanks for your explanation. I'm activating objects in NVDA by pressing NVDA+enter at the cursor position, and now bookmarks are very precise, that is, if my cursor is iat the begining of a paragraph, then I press NVDA+enter, then control+b, control+n and then I press enter on the bookmark, the cursor goes to the specific paragraph.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-995749257 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW4OSC5BHAVGYYTOYRRUBTURHIONANCNFSM5I3CGNPA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . https://github.com/notifications/beacon/ABW4OSFVH4HLLOZAPXSQG23URHIONA5CNFSM5I3CGNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHNM63CI.gif

[danielweck]

can sighted users use the Annotation feature? I don’t see a hot key for annotations.

Hello George, there is no annotation feature yet (although there has been a disabled menu item in the GUI pretty much from day one, not sure why ... perhaps a teaser (smile)

I have also suggested a hot key for “where am I.” RedShelf have a hot key and it provides the heading previous to where the RS thinks it is at and the page number. The pop up window is accessible and is open for about 10 seconds and closes, and one can hit esc to close it as well.

I have just merged a new feature which leverages the existing "publication info dialog", i.e. modal popup which can be invoked with CTRL + i keyboard shortcut. Just add the SHIFT modifier key and the focus will automatically be moved to the new info section which shows current page (if any, as per the page breaks authored in EPUB publications) and percentage progression in the current spine item. What is currently missing is a report of headings hierarchy extracted from the current document and some kind of mapping with the higher-level publication table of contents.

[nvdaes]

Hello George and Daniel: Thanks for this. I"ll edit my tests in Spanish, published on epubtest.org, making basic tests with the latest alpha, to mark the where I am feature as supported, in combination with NVDA list of elements dialog (NVDA f7), which reports the current heading. I"ll check if all this work. Also, the keyboard shortcuts may need to be updated in English and Spanish wiki, and also on nvdaes.github.io website, created by me and maintained also by other members of the community. If you don"t have any concern, I"ll do all this soon.

Enviado desde mi iPhone

El 22 dic 2021, a las 1:06, Daniel Weck @.***> escribió:

 can sighted users use the Annotation feature? I don’t see a hot key for annotations.

Hello George, there is no annotation feature yet (although there has been a disabled menu item in the GUI pretty much from day one, not sure why ... perhaps a teaser )

I have also suggested a hot key for “where am I.” RedShelf have a hot key and it provides the heading previous to where the RS thinks it is at and the page number. The pop up window is accessible and is open for about 10 seconds and closes, and one can hit esc to close it as well.

I have just merged a new feature which leverages the existing "publication info dialog", i.e. modal popup which can be invoked with CTRL + i keyboard shortcut. Just add the SHIFT modifier key and the focus will automatically be moved to the new info section which shows current page (if any, as per the page breaks authored in EPUB publications) and percentage progression in the current spine item. What is currently missing is a report of headings hierarchy extracted from the current document and some kind of mapping with the higher-level publication table of contents.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[nvdaes]

Hi again. Daniel, it would be great if the last positionwhen invoking control+shift+i was sabed, so we can resume reading without to create a new bookmark. Thanks

2021-12-22 4:34 GMT+01:00, Noelia Ruiz Martínez @.***>:

Hello George and Daniel: Thanks for this. I"ll edit my tests in Spanish, published on epubtest.org, making basic tests with the latest alpha, to mark the where I am feature as supported, in combination with NVDA list of elements dialog (NVDA f7), which reports the current heading. I"ll check if all this work. Also, the keyboard shortcuts may need to be updated in English and Spanish wiki, and also on nvdaes.github.io website, created by me and maintained also by other members of the community. If you don"t have any concern, I"ll do all this soon.

Enviado desde mi iPhone

El 22 dic 2021, a las 1:06, Daniel Weck @.***> escribió:



can sighted users use the Annotation feature? I don’t see a hot key for annotations.

Hello George, there is no annotation feature yet (although there has been a disabled menu item in the GUI pretty much from day one, not sure why ... perhaps a teaser )

I have also suggested a hot key for “where am I.” RedShelf have a hot key and it provides the heading previous to where the RS thinks it is at and the page number. The pop up window is accessible and is open for about 10 seconds and closes, and one can hit esc to close it as well.

I have just merged a new feature which leverages the existing "publication info dialog", i.e. modal popup which can be invoked with CTRL + i keyboard shortcut. Just add the SHIFT modifier key and the focus will automatically be moved to the new info section which shows current page (if any, as per the page breaks authored in EPUB publications) and percentage progression in the current spine item. What is currently missing is a report of headings hierarchy extracted from the current document and some kind of mapping with the higher-level publication table of contents.

—

Reply to this email directly, view it on GitHub, or unsubscribe.

You are receiving this because you were mentioned.

--

Reply to this email directly or view it on GitHub:

https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-999259736

You are receiving this because you are subscribed to this thread.

Message ID: @.***>

[danielweck]

Hello Noelia, I use the underscore special link to get back to the current reading position (one tab away from the keyboard shortcut which places the focus at the main position). Currently the focus after closing the popup modal goes to the originator info button which is situated very early in the toolbar, so reverse tab twice goes to the skip link which is another way to reach the main content area.

[nvdaes]

Hi Daniel, the underscore special link doesn't work with NVDA, at least for me, tested in 2 computers, after closing the modal dialog. Also, seems that control+i and control+shift+i, both keystrokes open the same dialog, which reports the percentage of the book.

2021-12-22 10:37 GMT+01:00, Daniel Weck @.***>:

Hello Noelia, I use the underscore special link to get back to the current reading position (one tab away from the keyboard shortcut which places the focus at the main position). Currently the focus after closing the popup modal goes to the originator info button which is situated very early in the toolbar, so reverse tab twice goes to the skip link which is another way to reach the main content area.

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-999428528 You are receiving this because you were mentioned.

Message ID: @.***>

[danielweck]

Hi Noelia, I am surprised to hear that the undescore link fails with NVDA, do you mean that it doesn't jump to the correct destination, which in the screen reader case is the reading location activated with the NVDA-enter key (or space key with VoiceOver)? Basically, the underscore link simply references the exact same place in the HTML document as the progression indicated in the modal info dialog.

Regarding the CTRL-i keyboard shortcut: this invokes the popup modal info dialog, with the default focus in the cross icon / close button. Adding the SHIFT modifier key lands directly on the progression section which is now the "where am I" feature is implemented. Hitting the ESCAPE key offers a quick way to discard the modal popup. The keyboard focus then resumes on the originator GUI button for the info popup dialog. I will see if there are technical ways to restore the keyboard focus directly into the HTML document after calling the "where am I" feature. Right now the underscore link is the only way (assuming the space/enter screen reader keyboard action was used beforehand to ensure that the position is synchronised)

[nvdaes]

Hi again Daniel, now I understand the control+shift+i feature and the difference respect control+i. The underscore link works when we move to a chapter from the TOC, create a bookmark, etc. It doesn't work if we press NVDA+enter and then control+shift+i to open the modal popup, and then Escape to close. Hope this helps

2021-12-22 15:26 GMT+01:00, Daniel Weck @.***>:

Hi Noelia, I am surprised to hear that the undescore link fails with NVDA, do you mean that it doesn't jump to the correct destination, which in the screen reader case is the reading location activated with the NVDA-enter key (or space key with VoiceOver)? Basically, the underscore link simply references the exact same place in the HTML document as the progression indicated in the modal info dialog.

Regarding the CTRL-i keyboard shortcut: this invokes the popup modal info dialog, with the default focus in the cross icon / close button. Adding the SHIFT modifier key lands directly on the progression section which is now the "where am I" feature is implemented. Hitting the ESCAPE key offers a quick way to discard the modal popup. The keyboard focus then resumes on the originator GUI button for the info popup dialog. I will see if there are technical ways to restore the keyboard focus directly into the HTML document after calling the "where am I" feature. Right now the underscore link is the only way (assuming the space/enter screen reader keyboard action was used beforehand to ensure that the position is synchronised)

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-999616066 You are receiving this because you were mentioned.

Message ID: @.***>

[danielweck]

Noelia, George, I have just realised during my tests that the VoiceOver key + space produces a click under the mouse cursor, which by chance resulted in the expected bookmarking result, but of course totally fails to deliver when the mouse cursor is elsewhere in Thorium's GUI. I also attempted to select text with VoiceOver just as I can do in my web browser Firefox, but this seems to fail in Thorium's Chromium embedded webview. This is annoying as I realise I have no way of creating precise bookmarks with VoiceOver (I was under a false impression before, due to coincidence of mouse cursor positioning!). I understand you are having more success with NVDA.

Otherwise, in today's good news: I have just added the headings trail feature in the "where am I" section of the publication info popup modal dialog. Use CTRL + SHIFT + i to get there quickly, although CTRL + i will of course work too, just with additional tabbing down the GUI. The condensed trail of headings gives an indication of heading level 0-6 hierarchy that "contains" the current reading location, and the expanded trail represents the entire preceeding sequence of headings that lead to the current reading location. The links are clickable and automatically close the modal dialog, and directly go into the publication document. Let me know what you think, please.

[nvdaes]

Hello Daniel, this is really exciting! I've tested it with the Fundamental Accessibility Tests ebook by DAISY, available in epubtest.org. But just h1 is presented when pressing control+shift+i. Here are steps to reproduce:

1. Open the book in Thorium.
2. Press h until nav-210 Navigate forward and backward through reflowed content, h2.
3. Press NVDA+enter to perform the action associated with the object where the review cursor is located, by default, the same position than the system cursor or browse mode cursor.
4. Press control+shift+i.
5. Expand the button. Here are results, in Spanish:

4/5 (Basic functionality) [50%] Página actual: Page 6 h1 Navigation Tests h1 Basic Functionality Tests h1 Navigation Tests

For me it should be possible to see trailing h2, not just h1.

Also, please, if you know how to test or purchase a divina book, I'd like to read it. Thanks and merry Christmas!

2021-12-23 3:10 GMT+01:00, Daniel Weck @.***>:

Noelia, George, I have just realised during my tests that the VoiceOver key

• space produces a click under the mouse cursor, which by chance resulted in the expected bookmarking result, but of course totally fails to deliver when the mouse cursor is elsewhere in Thorium's GUI. I also attempted to select text with VoiceOver just as I can do in my web browser Firefox, but this seems to fail in Thorium's Chromium embedded webview. This is annoying as I realise I have no way of creating precise bookmarks with VoiceOver (I was under a false impression before, due to coincidence of mouse cursor positioning!). I understand you are having more success with NVDA.

Otherwise, in today's good news: I have just added the headings trail feature in the "where am I" section of the publication info popup modal dialog. Use CTRL + SHIFT + i to get there quickly, although CTRL + i will of course work too, just with additional tabbing down the GUI. The condensed trail of headings gives an indication of heading level 0-6 hierarchy that "contains" the current reading location, and the expanded trail represents the entire preceeding sequence of headings that lead to the current reading location. The links are clickable and automatically close the modal dialog, and directly go into the publication document. Let me know what you think, please.

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-999990827 You are receiving this because you were mentioned.

Message ID: @.***>

[danielweck]

Hi Noelia, only a h1 is shown because that is the only heading that precedes the current reading location. The trail of headings depends on where your current reading location is in the document.

[danielweck]

Sorry, I hit enter too soon in my previous message, continuing: my suspicion is that the last saved reading location didn't trigger with NVDA-enter like it does when a user clicks on text with the mouse cursor. To eliminate this possibility, please try to navigate via the Table of Contents to a deeper level in the headings hierarchy (say an h2 or h3), then you will see the corresponding trail in the "where am I" section.

[danielweck]

Hello again Noelia, I have just tested Advanced-Accessibility-Tests-Mathematics-v1.2.4.epub and I now see that headings do not have DOM identifiers attributes, which makes them unreachable directly. Instead, the parent section has the required ID attribute. This seems to be a common authoring template so I am going to add support in Thorium. I will report back shortly.

[nvdaes]

Hi Daniel, I think that the problem may be that, if we are at level 2, the header presented is h1; if we are at level 3, I see h2, and I think that at least the previous heading should be presented. Seems that pressing NVDA+enter makes Thorium to know the current location. I'm trying with an ebook that has headings at level 2 and 3, and if I move the cursor to an h3, I see the previous h2 when I press control+shift+i. Hope this helps.

2021-12-23 7:47 GMT+01:00, Daniel Weck @.***>:

Hello again Noelia, I have just tested Advanced-Accessibility-Tests-Mathematics-v1.2.4.epub and I now see that headings do not have DOM identifiers attributes, which makes then unreachable directly. Instead, the parent section has the required ID attribute. This seems to be a common authoring template so I am going to add support in Thorium. I will report back shortly.

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000083401 You are receiving this because you were mentioned.

Message ID: @.***>

[nvdaes]

Hi again Daniel, sorry for my previous message. Now I see that when I move the cursor to an h4 section, h2 is presented in the info dialog, so maybe related to identifiers. Thanks

2021-12-23 8:46 GMT+01:00, Noelia Ruiz Martínez @.***>:

Hi Daniel, I think that the problem may be that, if we are at level 2, the header presented is h1; if we are at level 3, I see h2, and I think that at least the previous heading should be presented. Seems that pressing NVDA+enter makes Thorium to know the current location. I'm trying with an ebook that has headings at level 2 and 3, and if I move the cursor to an h3, I see the previous h2 when I press control+shift+i. Hope this helps.

2021-12-23 7:47 GMT+01:00, Daniel Weck @.***>:

Hello again Noelia, I have just tested Advanced-Accessibility-Tests-Mathematics-v1.2.4.epub and I now see that headings do not have DOM identifiers attributes, which makes then unreachable directly. Instead, the parent section has the required ID attribute. This seems to be a common authoring template so I am going to add support in Thorium. I will report back shortly.

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000083401 You are receiving this because you were mentioned.

Message ID: @.***>

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000108562 You are receiving this because you are subscribed to this thread.

Message ID: @.***>

[danielweck]

Thank you very much for reporting Noelia, I have now fixed the issue of headings without ID, by looking for the nearest ancestor sectioning element that is itself linkable. I tested the updated headings hierarchy scanning algorithm with various EPUBs, notably Advanced-Accessibility-Tests-Extended-Descriptions-v1.2.2.epub. Thorium is now building, should be ready to test soon.

[danielweck]

Noelia, for your information, I added this keyboard shortcut description to the English Wiki ( https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts ):

• SHIFT + CTRL + i (OpenReaderInfoWhereAmI): in reader window, publication info popup modal dialog with direct keyboard focus on section that displays the headings hierarchy trail and other contextual information for the current reading location, such as nearest preceding page break, as well as document index in the reading order (this is the "where am I" feature).

Spanish Wiki link: https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts-(Spanish)

[nvdaes]

Thanks Daniel:

I'll take care ASAP: I'll translate the shortcuts in the spanish wiki page and will upload them to the nvdaes.github.io website for a better reading in Spanish, since lang attribute on GitHub is set to English. Also, I'll test and update my results in epubtest.org.

2021-12-23 9:12 GMT+01:00, Daniel Weck @.***>:

Noelia, for your information, I added this keyboard shortcut description to the English Wiki ( https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts ):

• SHIFT + CTRL + i (OpenReaderInfoWhereAmI): in reader window, publication info popup modal dialog with direct keyboard focus on section that displays the headings hierarchy trail and other contextual information for the current reading location, such as nearest preceding page break, as well as document index in the reading order (this is the "where am I" feature).

Spanish Wiki link: https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts-(Spanish)

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000121097 You are receiving this because you were mentioned.

Message ID: @.***>

[GeorgeKerscher]

Dear Noelia and Daniel,

I am now using v1.7.4-alpha.1.1614906205

I go to a place in a book and press NVDA key +ender

I then press ctrl + shift + i

I am taken to the modal and can tell where I am at. I press b and expand the button.

I then press ctrl + end to go to the bottom of the navigation points in the book. I then activate the last link in that modal, and now I am back at the heading I was at when I started the where am I function.

I was thinking that esc would put me back where I was, but that leaves me up by the information button.

Do I have all of this correct?

Is there an easier way to return when you pressed shift + ctrl + i?

Best

George

From: Noelia Ruiz Martínez @.> Sent: Thursday, December 23, 2021 3:15 AM To: edrlab/thorium-reader @.> Cc: George @.>; Mention @.> Subject: Re: [edrlab/thorium-reader] Screen reader not automatically following hyperlinks destinations into content webview (from TOC GUI, etc.) (Issue #1594)

Thanks Daniel:

I'll take care ASAP: I'll translate the shortcuts in the spanish wiki page and will upload them to the nvdaes.github.io website for a better reading in Spanish, since lang attribute on GitHub is set to English. Also, I'll test and update my results in epubtest.org.

2021-12-23 9:12 GMT+01:00, Daniel Weck @. <mailto:@.> >:

Noelia, for your information, I added this keyboard shortcut description to the English Wiki ( https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts ):

• SHIFT + CTRL + i (OpenReaderInfoWhereAmI): in reader window, publication info popup modal dialog with direct keyboard focus on section that displays the headings hierarchy trail and other contextual information for the current reading location, such as nearest preceding page break, as well as document index in the reading order (this is the "where am I" feature).

Spanish Wiki link: https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts-(Spanish)

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000121097 You are receiving this because you were mentioned.

Message ID: @. <mailto:@.> >

— Reply to this email directly, view it on GitHub https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000192609 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW4OSAONB2R4Z3WS2AP6WLUSLZB7ANCNFSM5I3CGNPA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABW4OSEURTUIR2YVMB2R6YLUSLZB7A5CNFSM5I3CGNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHOO3UYI.gif Message ID: @. @.> >

[danielweck]

Hello George, thank you very much for testing. The escape key closes the modal popup dialog and restores the keyboard focus on the originating call site, which is by default the info icon in the top-right corner, so only a couple of key tabbing away from the hidden/self-revealing "skip to content" GUI link. From there you can tab once into the special underscore link which will take you back to the last known reading location. This location can be the destination of a navigation link from the table of contents (or bookmark, landmark, goto page, search result), although according to both you and Noelia, the NVDA+enter keyboard shortcut effectively activates the text currently read by the screen reader in the publication HTML document, which is great news as I am not able to find a similar trick with VoiceOver. Once again, CTRL-i is not much different from SHIFT-CTRL-i, the latter places the keyboard focus at the "where am I" reading progression section, but that's the only interaction difference. From there, if indeed you decide to click on one of the heading links in the trail, especially the last one in the list, then indeed you are likely to jump back into the document near your last reading location ... but not exactly, as the screen reader might have moved forward during its playback, without actually activating any of the text read aloud, which is why the NVDA-enter trick is essential. With VoiceOver I would like to know the method to achieve the same result: I want to be able to ask the screen reader to do the equivalent of mouse click in the text that is currently spoken. This will then activate the DOM fragment and will make it become the current reading location. For placing bookmarks ideally a range of text characters can be selected, which I managed to do with VoiceOver but the Chromium web browser engine was not registering this as a proper selection, I think it only existed in the screen reader perception of the web page structure.

[GeorgeKerscher]

Hi,

Using NVDA, I go to a paragraph or list item under a heading. I activate the object there with the NVDA + enter. I also route the mouse to that object and left mouse click.

I then check where am I and it gives me the correct location. I can return to that heading with the last link in the model or through the Magic Hidden Link , but I cannot get back to the exact paragraph or list item I was on. Still, this is pretty darn good.

MathML is still not working in this build.

Best

George

From: Daniel Weck @.> Sent: Thursday, December 23, 2021 12:16 PM To: edrlab/thorium-reader @.> Cc: George @.>; Mention @.> Subject: Re: [edrlab/thorium-reader] Screen reader not automatically following hyperlinks destinations into content webview (from TOC GUI, etc.) (Issue #1594)

Hello George, thank you very much for testing. The escape key closes the modal popup dialog and restores the keyboard focus on the originating call site, which is by default the info icon in the top-right corner, so only a couple of key tabbing away from the hidden/self-revealing "skip to content" GUI link. From there you can tab once into the special underscore link which will take you back to the last known reading location. This location can be the destination of a navigation link from the table of contents (or bookmark, landmark, goto page, search result), although according to both you and Noelia, the NVDA+enter keyboard shortcut effectively activates the text currently read by the screen reader in the publication HTML document, which is great news as I am not able to find a similar trick with VoiceOver. Once again, CTRL-i is not much different from SHIFT-CTRL-i, the latter places the keyboard focus at the "where am I" reading progression section, but that's the only interaction difference. From there, if indeed you decide to click on one of the heading links in the trail, especially the last one in the list, then indeed you are likely to jump back into the document near your last reading location ... but not exactly, as the screen reader might have moved forward during its playback, without actually activating any of the text read aloud, which is why the NVDA-enter trick is essential. With VoiceOver I would like to know the method to achieve the same result: I want to be able to ask the screen reader to do the equivalent of mouse click in the text that is currently spoken. This will then activate the DOM fragment and will make it become the current reading location. For placing bookmarks ideally a range of text characters can be selected, which I managed to do with VoiceOver but the Chromium web browser engine was not registering this as a proper selection, I think it only existed in the screen reader perception of the web page structure.

— Reply to this email directly, view it on GitHub https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000484322 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW4OSECNRJGYVMMHBMILT3USNYQRANCNFSM5I3CGNPA . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub . You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABW4OSHHI3WYO5CYAOQLSQLUSNYQRA5CNFSM5I3CGNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHORC3YQ.gif Message ID: @. @.> >

[nvdaes]

Hello, I agree with George, this is fantastic. Anyway, not always is possible to return to the previous heading. I used the non visual testing book, going to the Sidebar: The Blood-Brain Barrier section. This section is presented in the info dialog, but not as a link. I think it would be excellent if, after the headings, we can see the text of the clicked/activated object, since seems not all objects can be activated. Best

2021-12-23 22:21 GMT+01:00, George @.***>:

Hi,

Using NVDA, I go to a paragraph or list item under a heading. I activate the object there with the NVDA + enter. I also route the mouse to that object and left mouse click.

I then check where am I and it gives me the correct location. I can return to that heading with the last link in the model or through the Magic Hidden Link , but I cannot get back to the exact paragraph or list item I was on. Still, this is pretty darn good.

MathML is still not working in this build.

Best

George

From: Daniel Weck @.***>

Sent: Thursday, December 23, 2021 12:16 PM

To: edrlab/thorium-reader @.***>

Cc: George @.>; Mention @.>

Subject: Re: [edrlab/thorium-reader] Screen reader not automatically following hyperlinks destinations into content webview (from TOC GUI, etc.) (Issue #1594)

Hello George, thank you very much for testing. The escape key closes the modal popup dialog and restores the keyboard focus on the originating call site, which is by default the info icon in the top-right corner, so only a couple of key tabbing away from the hidden/self-revealing "skip to content" GUI link. From there you can tab once into the special underscore link which will take you back to the last known reading location. This location can be the destination of a navigation link from the table of contents (or bookmark, landmark, goto page, search result), although according to both you and Noelia, the NVDA+enter keyboard shortcut effectively activates the text currently read by the screen reader in the publication HTML document, which is great news as I am not able to find a similar trick with VoiceOver.

Once again, CTRL-i is not much different from SHIFT-CTRL-i, the latter places the keyboard focus at the "where am I" reading progression section, but that's the only interaction difference. From there, if indeed you decide to click on one of the heading links in the trail, especially the last one in the list, then indeed you are likely to jump back into the document near your last reading location ... but not exactly, as the screen reader might have moved forward during its playback, without actually activating any of the text read aloud, which is why the NVDA-enter trick is essential. With VoiceOver I would like to know the method to achieve the same result: I want to be able to ask the screen reader to do the equivalent of mouse click in the text that is currently spoken. This will then activate the DOM fragment and will make it become the current reading location. For placing bookmarks ideally a range of text characters can be selected, which I managed to do with VoiceOver but the Chromium web browser engine was not registering this as a proper selection, I think it only existed in the screen reader perception of the web page structure.

—

Reply to this email directly, view it on GitHub https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000484322 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW4OSECNRJGYVMMHBMILT3USNYQRANCNFSM5I3CGNPA .

Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub .

You are receiving this because you were mentioned. https://github.com/notifications/beacon/ABW4OSHHI3WYO5CYAOQLSQLUSNYQRA5CNFSM5I3CGNPKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOHORC3YQ.gif Message ID: @. @.> >

--

Reply to this email directly or view it on GitHub:

https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000528491

You are receiving this because you were mentioned.

Message ID: @.***>

[nvdaes]

Hi Daniel:

I'm trying to clone the wiki to attach the spanish wiki page with latest changes, but I get this error:

warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'

I'll try again when you can fix this. Merry Christmas!

2021-12-23 9:12 GMT+01:00, Daniel Weck @.***>:

Noelia, for your information, I added this keyboard shortcut description to the English Wiki ( https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts ):

• SHIFT + CTRL + i (OpenReaderInfoWhereAmI): in reader window, publication info popup modal dialog with direct keyboard focus on section that displays the headings hierarchy trail and other contextual information for the current reading location, such as nearest preceding page break, as well as document index in the reading order (this is the "where am I" feature).

Spanish Wiki link: https://github.com/edrlab/thorium-reader/wiki/Keyboard-shortcuts-(Spanish)

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1000121097 You are receiving this because you were mentioned.

Message ID: @.***>

[danielweck]

George, Noelia, FYI I reported in another issue ( https://github.com/edrlab/thorium-reader/issues/1521#issuecomment-1001041853 ) about a new SHIFT CTRL "n" keyboard shortcut which opens the navigation panel (like CTRL "n") but also directly focuses on the current heading (there is also some visual indication of the headings in the current HTML chapter / spine item, and a strong visual emphasis style on the heading closest to the current reading location).

Side note, in response to your previous feedback: in my tests without screen reader, I also experienced "hit and miss" mouse click on paragraphs to set the current reading location, and see it reflected in the underscore special link at the top of the HTML document. I will look into this at the next release. Right now reliability is not satisfactory.

As for MathML, I do not know what caused the breaking change (MathJax is the same version, and the problem seem to occur when MathJax is turned off anyway). I will investigate next year (smile).

Merry Christmas / Happy Holidays.

[nvdaes]

Thanks Daniel. When you can, please look at the issue about the wiki. In the meantime, I"ll update the spanish webpage about Thorium at nvdaes.github.io and my results in epubtest.org, probably tomorrow. Merry Christmas!

Enviado desde mi iPhone

El 25 dic 2021, a las 17:28, Daniel Weck @.***> escribió:

 George, Noelia, FYI I reported in another issue ( #1521 (comment) ) about a new SHIFT CTRL "n" keyboard shortcut which opens the navigation panel (like CTRL "n") but also directly focuses on the current heading (there is also some visual indication of the headings in the current HTML chapter / spine item, and a strong visual emphasis style on the heading closest to the current reading location).

Side note, in response to your previous feedback: in my tests without screen reader, I also experienced "hit and miss" mouse click on paragraphs to set the current reading location, and see it reflected in the underscore special link at the top of the HTML document. I will look into this at the next release. Right now reliability is not satisfactory.

As for MathML, I do not know what caused the breaking change (MathJax is the same version, and the problem seem to occur when MathJax is turned off anyway). I will investigate next year (smile).

Merry Christmas / Happy Holidays.

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.

[danielweck]

Ah, I found a useful VoiceOver keyboard combo: 1) Ctrl + Option + Command + F5 (on my laptop I also have to press the Function key in order to toggle F5, and because of the so-called Touch Bar I must tap once with my finger on the smooth tactile area above the physical keyboard, then double-tap again to actually trigger the F5 key ... very cumbersome interaction!) Anyway, this brings the pointing device cursor (e.g. mouse, trackpad) over VoiceOver's own playback cursor. 2) Shift + Control + Option + Space which triggers a click event where the mouse cursor is, causing Thorium to record the current reading location.

Apple Voice Over documentation for cursor tracking: https://support.apple.com/en-gb/guide/voiceover/vo15534/mac

[danielweck]

I have also discovered that Voice Over Control + Option + Space doesn't exactly equate to hitting the enter key on a hyperlink. This is noticeable with the underscore special link (the feature is broken because of this), so I am implementing a fix / workaround.

[danielweck]

Hello, I implemented a workaround for the screen reader (well, at least MacOS Voice Over) handling of link activation via enter key versus other keystrokes that cause mouse interaction (this in turn created false positives when recording the current reading location, and it corrupted the underscore special link). In other words, the underscore special link should now be much more reliable after using the screen reader's technique to mouse-click on DOM text in order to activate the current reading location, which can then be bookmarked, or simply used via the underscore link to resume reading after navigating to some GUI such as the settings panel or the publication information popup ("where am I" feature). Note: I am aware that the bookmark feature is very inaccessible at the moment, and I also experience unreliable CTRL-B for create and remove bookmarks. I will investigate.

[danielweck]

Ah, bad news, VoiceOver's text selection feature with the VO keys + Enter only works in Safari. It completely fails in Firefox, and appears to initially work in Chrome but fails at actually creating the text selection in the webview. Thorium is an Electron application therefore built with the Chromium web browser engine, where I observe the same behaviour.

So, I managed to improve the bookmark add/delete logic when invoked via the user interface icon button in the toolbar (to toggle bookmark at the entire page level), as well as when using the CTRL-B keyboard shortcut to add fine-grain bookmarks at the precise current reading location (optionally with selected text in order to set the bookmark title).

Perhaps you will have more success with Narrator, JAWS and NVDA to select character ranges in the HTML document in order to create bookmarks with the name inferred from the selected text. Please let me know. You will hear the screen reader speak an ARIA live alert notification that provides the action (add or delete) and the total number of bookmarks recorded.

[nvdaes]

Hi Daniel, I've tested pressing NVDA+enter in NVDA after your workaround and I can move to the saved location after closing the info dialog (control+shift+i), or in other circumstances, for example, pressing m to go to the special underscore link. I'm super-impressed with this! This is really amazing!

I want to update at least the spanish webpage about Thorium at nvdaes.github.io, and the wiki when you can fix it. Thanks a lot!

2021-12-26 19:28 GMT+01:00, Daniel Weck @.***>:

Hello, I implemented a workaround for the screen reader (well, at least MacOS Voice Over) handling of link activation via enter key versus other keystrokes that cause mouse interaction (this in turn created false positives when recording the current reading location, and it corrupted the underscore special link). In other words, the underscore special link should now be much more reliable after using the screen reader's technique to mouse-click on DOM text in order to activate the current reading location, which can then be bookmarked, or simply used via the underscore link to resume reading after navigating to some GUI such as the settings panel or the publication information popup ("where am I" feature). Note: I am aware that the bookmark feature is very inaccessible at the moment, and I also experience unreliable CTRL-B for create and remove bookmarks. I will investigate.

-- Reply to this email directly or view it on GitHub: https://github.com/edrlab/thorium-reader/issues/1594#issuecomment-1001225126 You are receiving this because you were mentioned.

Message ID: @.***>

[danielweck]

Hello @nvdaes Noelia!

I'm trying to clone the wiki to attach the spanish wiki page with latest changes, but I get this error: warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'

I am afraid this is not a GitHub / Wiki problem, but something wrong at your end, in your Git clone. I assume you called the git clone command on the URL https://github.com/edrlab/thorium-reader.wiki.git I have just tried, it works fine at my end. By default the git clone command line instruction creates a folder named thorium-reader.wiki, perhaps there is a conflict on your Windows filesystem (i.e. existing folder?) Is it the first time you attempt cloning the Wiki repository, or are you trying to update an existing clone?

[nvdaes]

Hello Daniel, I am using the git clone command with the corresponding URL. As far as I know, nothing should create a conflict since I didn"t have a folter with this repo. I"ll try to clone the wiki to a folder not containing a dot or something, just for testing. I"ll look at hthis in the next days.

Enviado desde mi iPhone

El 30 dic 2021, a las 15:11, Daniel Weck @.***> escribió:

 Hello @nvdaes Noelia!

I'm trying to clone the wiki to attach the spanish wiki page with latest changes, but I get this error: warning: Clone succeeded, but checkout failed. You can inspect what was checked out with 'git status' and retry with 'git restore --source=HEAD :/'

I am afraid this is not a GitHub / Wiki problem, but something wrong at your end, in your Git clone. I assume you called the git clone command on the URL https://github.com/edrlab/thorium-reader.wiki.git I have just tried, it works fine at my end. By default the git clone command line instruction creates a folder named thorium-reader.wiki, perhaps there is a conflict on your Windows filesystem (i.e. existing folder?) Is it the first time you attempt cloning the Wiki repository, or are you trying to update an existing clone?

— Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.