Keyboard shortcuts

[gregoriopellegrino]

OS: Windows 10 Assistive technology: NVDA 2018.1.1

It is quite difficult to move in the UI between panels (TOC, pages, content, header): it would be very useful to implement the keyboard shortcut F6, that is "native" in Windows.

[danielweck]

Note: in Readium "1" (cloud / web reader and Chrome app) a number of keyboard interactions / shortcuts are implemented in order to facilitate key-driven UI navigation. Documentation: https://github.com/readium/readium-js-viewer/wiki/Keyboard

[llemeurfr]

Reading the R1 document, I see mention of Windows function keys only where stated that F8 is not supported. So, first, should we support Windows function keys here ? then what is the list of function keys we should support, for which action (for which assistive technology)?

[llemeurfr]

From what I find on the Web, we could map some Windows function keys to useful actions:

F1 | Brings up a Help window (but we don't have a Help so far) F3 | Opens search box in browsers (could move to the search field in the bookshelf view, open a search box in the reading view) F6 | Cycles through the screen elements in a window (see Gregorio's request) F10 | Activates menu bar options (use to be discussed) F11 | Toggles between full screen and normal display (could be a trigger to the Zen mode in the reading view)

[llemeurfr]

From George's comments on the use of Thorium vs Daisy tests, it seems we are lacking a keyb shortcut to move between chapters without going through the ToC.

[llemeurfr]

Only F6 seems really interesting in practice. But not easy to implement. Slack has this feature.

F3: people use Ctrl-F in practice, and it does not seem really useful in the bookshelf view as the search field is always displayed. It will be useful for implementing search inside a book, later.

[panaC]

How to do implement the F6 feature ?

The F6 allow to focus on each main screen block at each hit

[danielweck]

Note that F3 / CTRL-F (and the "reverse" variant using the SHIFT key modifier) can be used to cycle through search results too, depending on the context. So this is not just about quick-access to a text input field / search box.

[danielweck]

F6 (and SHIFT + F6) is about cycling through GUI landmarks. Note that screen readers typically layer additional logical navigation, based on structural-semantics in the HTML markup (e.g. heading levels, ARIA roles, etc.).

[danielweck]

Common NVDA & JAWS Keystrokes

(contributed by Prashant Verma, DAISY Consortium)

For Reading Text, navigation in tables, navigation in browser window Insert key can be replaced with the screen reader modifier key (e.g. caps lock)

Description	Command
Say Prior Character	LEFT ARROW
Say Next Character	RIGHT ARROW
Prior Word	CTRL+LEFT ARROW
Next Word	CTRL+RIGHT ARROW
Prior Line	UP ARROW
Next Line	DOWN ARROW
Say Prior Sentence	ALT+UP ARROW (Jaws ONLY)
Say Next Sentence	ALT+DOWN ARROW (Jaws ONLY)
Say All	INSERT+DOWN ARROW
Say Font	INSERT+F

Tables

Description	Command
Cell to Right	ALT+CTRL+RIGHT ARROW
Cell to Left	ALT+CTRL+LEFT ARROW
Cell Below	ALT+CTRL+DOWN ARROW
Cell Above	ALT+CTRL+UP ARROW

Quick keys for navigation in the browser

The screen reader needs to be in the browse or virtual cursor mode

Description	Command
Next heading	h
Next table	t
Next graphic	g
Next list	l
Next form field	f
List of elements	Insert+F7 (works irrespective of the cursor mode)

Use the above keys with Shift to move to the previous element

[danielweck]

Imported from https://github.com/readium/readium-desktop/issues/572 Keyboard shortcuts for changing text size TODO: study the keyboard shortcut which could trigger that info

[danielweck]

Note that at the time of writing, Thorium implements the following keyboard shortcuts in the reader window: 1) LEFT/RIGHT arrows, when keyboard focus in user interface (but not inside popup modal dialogs and menus) or inside EPUB document (but not inside text input fields or other interactive HTML elements) => "page turn" in paginated mode, or "page scroll" in scroll mode. 2) LEFT/RIGHT arrows with key modifiers CTRL + SHIFT (on MacOS) with additional ALT (on Windows and Linux) => navigate previous/next spine items (aka "chapters")

[danielweck]

Also note that at the time of writing, Thorium implements the following keyboard shortcuts in the library window (catalog section, OPDS feeds):

• LEFT/RIGHT arrows with key modifiers CTRL + SHIFT (on MacOS, Windows and Linux) => OPDS pagination previous/next

[llemeurfr]

On JAWS / Windows,

• CTRL + SHIFT + ALT does not move from chapter to chapter; we get an error message "you are not in a table".
• UP and DOWN move from line to line (good) but I don't find a way to read a whole content of a chapter.

How should it work?

[danielweck]

With every screen reader there is a "start continuous reading" from current position. For NVDA I printed this PDF: https://dequeuniversity.com/screenreaders/nvda-keyboard-shortcuts

[danielweck]

For JAWS: https://dequeuniversity.com/screenreaders/jaws-keyboard-shortcuts

[danielweck]

Usually SPECIAL_KEY + arrow-down (I use CAPS LOCKS, as I don't have INSERT)

[danielweck]

For JAWS, simply change this code (for example remove ALT, like we already do for MacOS VoiceOver): https://github.com/readium/readium-desktop/blob/80dbecabf4d8ea453c0917bb590020b32655744d/src/renderer/components/reader/Reader.tsx#L375-L376 ...and see if JAWS lets the keyboard shortcut pass through its own keystroke filter.

[ways2read]

Some suggested keyboard shortcuts. I am thinking here of Windows:

Ctrl + f4 (and maybe Ctrl + W and Escape) = close window, back to the bookshelf Ctrl + I = information ? = detach window Ctrl + B = toggle Bookmark (there is a duplication below) Ctrl + S = Settings (Edge is ctrol + alt + o) Ctrl + N = Navigation

Items like TOC and go to page are within the navigation control, so I don’t know if it is possible to go straight to them, but if you can: Ctrl + T = TOC (Edge is Alt + T) Ctrl + L = Landmarks Ctrl + B = Bookmarks (Edge is Alt + B, where you add/delete and navigate bookmarks) Ctrl + A = Annotations Ctrl + G = Go to page (same as Edge)

Daniel also wondered about a ‘go to the main reading area’ key.

Text smaller/bigger = ctrl + ‘-‘/'+' (same as Edge and many others) Forward/back a screen (in scrolling mode it moves the screen down/up) = PgDn/PgUp (common Windows key) Forward/back a document = Ctrl + PgDn/PgUp (common Windows key, same as VitalSource Bookshelf) Toggle full screen = F11

Thinking ahead: Ctrl + F = Find (same as Edge) Ctrl + R = read aloud (missing shortcut in Edge!)

Getting to the Where am I? information is a common challenge, getting to it without losing the place in the text. Perhaps ctrl + i can also include the location information? Or ctrl _ shift + i?

[danielweck]

FYI: I am finalizing PR https://github.com/readium/readium-desktop/pull/958

[danielweck]

The current set of keyboard shortcuts is described in this Wiki page: https://github.com/readium/readium-desktop/wiki/Keyboard-shortcuts

[ways2read]

I am in the reader view, in the main content window enjoying my book with NVDA. I want to go back to my library (ideally there would be a shortcut for this), I press ctrl T and the focus moves to the top, but NVDA says 'blank'. I tab once, the button appears 'Skip to content' but there is no speech. I press again and I hear 'Back to the bookshelf'.

Could the top landmark be labelled so it is spoken? Can you fix the silent 'Skip to content button'?

[GeorgeKerscher]

-keyboard shortcuts using NVDA version 19.3 on Windows 10.

Most of theshortcut keys worked as expected, except for these: -Ctrl+B inserts a bookmark. When I go to that bookmark, I am at the beginning of the spine item where I left the bookmark. -I changed the next spine item to CTRL+PageDown -I changed the Previous spine item toCtrl+PageUp -I love the ability to change the hot keys, and the interface is great.

-left or right arrow moves me to the next character. Ctrl+right arrow moves me to the next word, which is what is expected with a screen reader. -Ctrl+F in library view opens, but it is unclear how this function works, i.e. what is the result supposed to do?

[GeorgeKerscher]

Ctrl+T and CTRL Home do the same thing. Using a screen reader, CTRL+Home takes you to the top of the page. It seems that CTRL+T is a shortcut that could take a person to the table of contents.

[danielweck]

I want to go back to my library (ideally there would be a shortcut for this)

There is now CTRL + w on all platforms (Windows, Mac and Linux), which unlike META + w on Mac (i.e. using the Apple / Command modifier key) does not exit the app when closing the reader window, but instead ensures that the library window is restored.

[danielweck]

Could the top landmark be labelled so it is spoken? Can you fix the silent 'Skip to content button'?

This was fixed via https://github.com/readium/readium-desktop/commit/afa682796765c9c3877fd2496b272e8ef0362767

[danielweck]

-I changed the next spine item to CTRL+PageDown -I changed the Previous spine item toCtrl+PageUp

This is actually now a built-in feature in Thorium. I implemented support for "alternative" keyboard shortcuts for situations such as when arrow keys are effectively reserved by screen readers for their own top-level interactions.

Note: if you have previously edited / customized your own keyboard shortcut scheme, you may need to reset the default preset using the dedicated menu in Thorium's settings section.

See the full description in the wiki page: https://github.com/readium/readium-desktop/wiki/Keyboard-shortcuts

[ways2read]

Confirming that the ctrl+w works great on my Windows 10+NVDA combination. Also the top landmark and skip to content are spoken- good job!

Ctrl+PgUp and Ctrl+PgDn move to the next spine item, but the reading position is left in a strange place, Like on the same place on the page, but in the new document. I think that ideally the reading position will be the top of the new document.

[danielweck]

-Ctrl+B inserts a bookmark. When I go to that bookmark, I am at the beginning of the spine item where I left the bookmark.

I'm not sure which revision of Thorium you used, but there is a fix for this problem in the develop branch (therefore available in the automated releases https://github.com/readium/readium-desktop/releases ).

This fix is actually a pretty big leap forward, because it applies to any kind of content navigation, e.g. TOC or bookmark links from the navigation panel (Thorium's GUI), but also hyperlinks clicked directly from within EPUB documents, etc.

The fix is actually a workaround which consists in "instructing" the screen reader to focus on a particular HTML element even if this element is not focusable, let alone keyboard-tabbable (note to developers: this is achieved using tabindex with -1 value, as well as a temporary visual clue to indicate link destination, much like CSS :target pseudo-class).

Now, the caveat is that because EPUB HTML documents are rendered within their own sandboxed webview / frame, the screen reader doesn't necessarily follows deep into the content. For example, Voice Over remains outside the frame container (just "above" it, where the "main" landmark is), whereas NVDA pauses but then automatically dives into the linked content (which is neat!). So, for Voice Over, and just for general usability, there is now a special link inserted right at the beginning of EPUB HTML documents, which (much like the "skip to content" link in the GUI) acts as a redirector to area of interest, in this case: the currently-linked anchor in EPUB documents (for example, the last TOC destination that was clicked, or bookmark, or current reading location, or followed hyperlink within HTML documents).

In other words, the UX with VoiceOver is to activate a TOC or bookmark link, then to wait briefly for the keyboard focus to be automatically placed immediately before the frame container (at which point NVDA kindly dives deep into the content), and as Voice Over stays there, simply tab once in order to get access inside the frame, directly on the first hyperlink which points to the exact link location (e.g. a particular chosen section from the TOC).

Please try and let me know how it goes. This is a pretty heavyweight screen reader workaround, but so far the best I have found using trial and error (the main issue to solve being the transition from GUI to EPUB content inside the frame).

[danielweck]

-Ctrl+F in library view opens, but it is unclear how this function works, i.e. what is the result supposed to do?

The CTRL + f command simply redirects keyboard focus to the existing search input text field in the GUI (and automatically selects the text that is present in the input field, just like CTRL + a does).

Eventually, this command may be used in the reader window's TOC (i.e. when a search feature is implemented there too), same with the bookmarks list, and of course the full text search inside publication contents. Right now, the search feature is only available for publication titles in the local library window, and also the OPDS search which is on the server side (remote content repository with its own search heuristics).

Note that eventually, CTRL + F3 (with optional SHIFT reverse key modifier) might be used for the search next / previous commands.

[danielweck]

Ctrl+T and CTRL Home do the same thing. Using a screen reader, CTRL+Home takes you to the top of the page. It seems that CTRL+T is a shortcut that could take a person to the table of contents.

If I understand correctly, CTRL + home is a screen reader feature (it is certainly not implemented by Thorium). It is also probably specific to a particular screen reader (NVDA, I guess?). There are a number of key combinations that do not work because of either system-level functionality (e.g. F4 in Mac OS) and of course the screen reader commands which take precedence over anything implemented at the application level. I chose CTRL + t because in English "T" is for "Toolbar", but I am happy to follow your advice, as long as the change works consistently on Mac, Windows and Linux.

[danielweck]

Ctrl+PgUp and Ctrl+PgDn move to the next spine item, but the reading position is left in a strange place, Like on the same place on the page, but in the new document. I think that ideally the reading position will be the top of the new document.

Did you start NVDA before launching Thorium? This is important, because the Electron framework and the Chromium webview subsystem (which Thorium is based on) activate a special accessibility bridge when a screen reader is detected (apparently, this is because of significant performance loss when the screen reader layer is activated unnecessarily). In the specific case of Thorium (or for that matter, any app built on top of Readium's Electron "navigator" component), we actually go one step further by ensuring that EPUB content documents are rendered in completely fresh frames (internally: cleanly-reloaded webviews) every time a link is followed (e.g. TOC, bookmark, etc.). This introduces a small delay but based on my tests with NVDA and VoiceOver this is a necessary evil, because otherwise the screen reader buffer continues to read the old DOM (even though it has been completely garbage-collected by the underlying Electron / Chromium components).

[danielweck]

PS: at this point I should come clean about the keys "page up/down" and "home" ... my primary keyboard does not actually support these keys (i.e. no physical buttons, and no virtual "function" key modifier to emulate the key code), so I tend to ignore these keys in my design decisions. Naturally, I added support for the alternative "page up/down" shortcuts because it made a lot of sense for inter-chapter navigation, but I haven't actually tested this myself (I just use the primary arrow keys shortcuts instead). So, your feedback is greatly appreciated :)

[GeorgeKerscher]

Previous and next spine item shortcut seem backward. IMO PageUp moves forward (previous) in the book and PageDown (next) seems correct. What is there now is the opposite of how I view this.

[GeorgeKerscher]

Hello,

I have the alpha 426. I have the same problem. I insert a bookmark in a chapter somewhere, normally several heading s down. I then navigate away to a different chapter. I go to the bookmarks, and I get taken to the beginning of the chapter.

Best

George

From: Daniel Weck notifications@github.com Sent: Wednesday, March 11, 2020 4:16 AM To: readium/readium-desktop readium-desktop@noreply.github.com Cc: George kerscher@montana.com; Manual manual@noreply.github.com Subject: Re: [readium/readium-desktop] Keyboard shortcuts (#155)

-Ctrl+B inserts a bookmark. When I go to that bookmark, I am at the beginning of the spine item where I left the bookmark.

I'm not sure which revision of Thorium you used, but there is a fix for this problem in the develop branch (therefore available in the automated releases https://github.com/readium/readium-desktop/releases ).

This fix is actually a pretty big leap forward, because it applies to any kind of content navigation, e.g. TOC or bookmark links from the navigation panel (Thorium's GUI), but also hyperlinks clicked directly from within EPUB documents, etc.

The fix is actually a workaround which consists in "instructing" the screen reader to focus on a particular HTML element even if this element is not focusable, let alone keyboard-tabbable (note to developers: this is achieved using tabindex with -1 value, as well as a temporary visual clue to indicate link destination, much like CSS :target pseudo-class).

Now, the caveat is that because EPUB HTML documents are rendered within their own sandboxed webview / frame, the screen reader doesn't necessarily follows deep into the content. For example, Voice Over remains outside the frame container (just "above" it, where the "main" landmark is), whereas NVDA pauses but then automatically dives into the linked content (which is neat!). So, for Voice Over, and just for general usability, there is now a special link inserted right at the beginning of EPUB HTML documents, which (much like the "skip to content" link in the GUI) acts as a redirector to area of interest, in this case: the currently-linked anchor in EPUB documents (for example, the last TOC destination that was clicked, or bookmark, or current reading location, or followed hyperlink within HTML documents).

In other words, the UX with VoiceOver is to activate a TOC or bookmark link, then to wait briefly for the keyboard focus to be automatically placed immediately before the frame container (at which point NVDA kindly dives deep into the content), and as Voice Over stays there, simply tab once in order to get access inside the frame, directly on the first hyperlink which points to the exact link location (e.g. a particular chosen section from the TOC).

Please try and let me know how it goes. This is a pretty heavyweight screen reader workaround, but so far the best I have found using trial and error (the main issue to solve being the transition from GUI to EPUB content inside the frame).

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/readium/readium-desktop/issues/155?email_source=notifications&email_token=ABW4OSFYC5SNYJLJO33OTZDRG5QFVA5CNFSM4FCS2VSKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEOO6EUQ#issuecomment-597549650 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW4OSBCBRRVCPBKC74DH4DRG5QFVANCNFSM4FCS2VSA .

[danielweck]

Previous and next spine item shortcut seem backward. IMO PageUp moves forward (previous) in the book and PageDown (next) seems correct. What is there now is the opposite of how I view this.

No problems, I will swap them around.

[danielweck]

I insert a bookmark in a chapter somewhere, normally several heading s down. I then navigate away to a different chapter. I go to the bookmarks, and I get taken to the beginning of the chapter.

Ah, yes, this is a tricky general problem which is currently unresolved. Basically, a screen reader moves its "focus" (i.e. reading location) outside of the visible viewport of the HTML document, but unfortunately the DOM does not receive notifications (e.g. such as "scroll" events) that would allow the application to track the screen reader's reading position, which would then enable the application to bring the correct HTML elements into view (i.e. to synchronize the application's own record of the reading location, based on what is visually shown on the screen, either in paginated mode or via a more traditional scroll view).

George, using your screen reader, are you able to generate a mouse click inside the currently-narrated HTML element? If so, then Thorium should be able to detect this "virtual" mouse click, which will then set the reading location to this exact spot. Then (after a few milliseconds), just hit CTRL + b in order to place a bookmark there.

[danielweck]

Regarding placing the mouse cursor and then clicking onto the HTML element that the screen reader is currently reading aloud, I tested on Mac OS with Voice Over, using the default VO "activation key" which is CTRL + OPTION (quite a mouthful):

1) CTRL OPTION (VO) + COMMAND F5 (to place the cursor over the HTML element) 2) CTRL OPTION (VO) + SHIFT SPACE (to click where the cursor is located)

Unfortunately, the problem remains, because the click may in fact occur in an unexpected / incorrect area of the application user interface, due to the webview not scrolling automatically to where the screen reader is currently talking. The same problem exists in a regular web browser, in cases where a few pixels of the paragraph being read are visible, but overall the text content is hidden / overflows beyond the visible viewport.

The net result is that the "click on HTML element" technique cannot be reliably used to set a bookmark on an accurate location in the publication document.

[GeorgeKerscher]

Hello,

What about highlighting some text and even copying it to the clipboard? This might be more like an annotation, but if it gives us focus, that could work. This may have an added benefit of the bookmarks getting text associated with it instead of just a number, which is difficult to track if you have a bunch.

Best

George

From: Daniel Weck notifications@github.com Sent: Thursday, March 12, 2020 2:24 AM To: readium/readium-desktop readium-desktop@noreply.github.com Cc: George kerscher@montana.com; Manual manual@noreply.github.com Subject: Re: [readium/readium-desktop] Keyboard shortcuts (#155)

I insert a bookmark in a chapter somewhere, normally several heading s down. I then navigate away to a different chapter. I go to the bookmarks, and I get taken to the beginning of the chapter.

Ah, yes, this is a tricky general problem which is currently unresolved. Basically, a screen reader moves its "focus" (i.e. reading location) outside of the visible viewport of the HTML document, but unfortunately the DOM does not receive notifications (e.g. such as "scroll" events) that would allow the application to track the screen reader's reading position, which would then enable the application to bring the correct HTML elements into view (i.e. to synchronize the application's own record of the reading location, based on what is visually shown on the screen, either in paginated mode or via a more traditional scroll view).

George, using your screen reader, are you able to generate a mouse click inside the currently-narrated HTML element? If so, then Thorium should be able to detect this "virtual" mouse click, which will then set the reading location to this exact spot. Then (after a few milliseconds), just hit CTRL + b in order to place a bookmark there.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/readium/readium-desktop/issues/155#issuecomment-598063397 , or unsubscribe https://github.com/notifications/unsubscribe-auth/ABW4OSHYM6UZSWIZNWZVQCDRHCL3LANCNFSM4FCS2VSA .