Brltty and NVDA Interaction; only some buttons on Braille display recognized as NVDA input gestures.

[nvaccessAuto]

Reported by DebeeArmstrong on 2014-06-05 17:04 TeleSensory PowerBraille 40 (PB40) works beautifully under Brltty in Linux. In NVDA, it displays and scrolls fine, but either NVDA, Brltty, Brlapi or the TSI display driver in Brltty are not recognizing all the buttons.

This might need to be fixed upstream, or perhaps there's a user table that can be edited. Further documentation would be appreciated. Perhaps a wiki article on how Brltty, Brlapi, the individual Braille display drivers and NVDA all co-exist.

See my related documentation bug: #4176 (Invalid (depricated) link in documentation.) http://community.NVDA-project.org/ticket/4176#ticket

I tested with three separate PB40s on three different Windows 7 computers, all connected to hardware (not USB) serial ports. The results were identical. I also switched displays so each computer was tested once with each different display.

In most cases, a button was seen by NVDA but not assignned, so I assigned it myself under input gestures.

To test, I got the Brltty button names with input help Insert-1. I also used Preferences-Input gestures, to assign as many of the buttons as NVDA could detect when prompted to enter an input gesture.

The remainder of this report contains a list of the buttons that do work properly, followed by a list of those that either are not recognized or are only partly seen by NVDA.

In the below list I give the Brltty button name as reported by NVDA followed with the assignment, followed by what physical button(s) it's mapped to on my PB40.

Buttons it does recognize: Brltty LNUP: moves display to prev line; mapped to right rocker up Brltty lndn: moves display to next line; mapped to right rocker down Brltty prefmenu: assigned myself successfully; mapped to pressing right and left buttons together Brltty csr_jmpvert: assigned myself successfully; mnapped to pressing left and right rockers down together Brltty Home: assigned myself successfully; mapped to concave button Brltty csrtrk: assigned myself successfully; mapped to convex button Brltty freeze: assigned myself successfully; mapped to pressing concave and convex buttons together Brltty mute: assigned myself successfully; mapped to right rocker up and right button pressed together Brltty lnbeg: assigned myself successfully; mapped to left button and left rocker up pressed together Brltty say_line: assigned myself successfully; mapped to right button and right rocker down pressed together Brltty LNMED: assigned myself successfully; mapped to right button and left rocker up pressed together Brltty top_left: assigned myself successfully; mapped to left button and right rocker pressed together Brltty bot_left: assigned myself successfully; mapped to left button and right rocker down pressed together Brltty Paste: assigned myself successfully; mapped to left rocker down and concave pressed together Brltty Back: assigned myself successfully; mapped to left rocker up and concave pressed together Brltty windn: assigned myself successfully; mapped to left rocker down and convex pressed together Brltty winup: assigned myself successfully; mapped to left rocker up and convex pressed together Brltty prdifly: assigned myself successfully; mapped to right rocker down and concaved pressed together Brltty nxdifly: assigned myself successfully; assigned to right rocker down and concave pressed together

In this list, I've given button names where NVDA could report them. In most cases, it didn't see the button, so I only report the physical button on the device.

buttons not recognized: Brltty fwinlt: scrolls braille back; should be assigned to the left button, could not assign myself Brltty fwinrt: scrolls display forward; should be assigned to the right button, could not assign myself PB left rocker up: not seen by NVDA at all; cannot assign Pb left rocker down: not seen by NVDA at all; cannot assign PB left and right rockers pressed up together: not seen at all by NVDA; cannot assign PB Left button and left rocker down pressed together: not seen at all by NVDA; cannot assign PBRight button and left rocker down pressed together: not seen at all by NVDA; cannot assign

The left and right buttons, fwinlt and fwinrt respectively, were listed in the tree view for Braille input gestures for scrolling Braille back and forward. They were also recognized when attempting to assign them with the Add button. However they never performed any assigned function and were not seen by input help. Removing and adding them again put them back in the list of input gestures, but NVDA could not actually see these buttons being pressed on the display.In other words, NVDA could only see these buttons being pressed when actually in the Input gestures dialog, when user is prompted to enter an input gestures.

The large left rocker bar, when pressed up and down without pressing any other buttons, was never recognized by NVDA though Brltty in Linux and other screen readers see it , and its function can easily be reassigned with other screen readers.

[nvaccessAuto]

Comment 1 by DebeeArmstrong on 2014-06-05 17:22 This is version 4.5-1 of Brltty. The BrlApi dll included shows version 0.6 in the Windows properties. The download site for Brltty compiled for Windows has an enormous number of versions and it's difficult to tell which is current; this appeared to be the correct one. It was downloaded from http://brl.thefreecat.org/brltty/

[nvaccessAuto]

Comment 2 by jteh (in reply to comment description) on 2014-06-06 04:04 Thanks for the report. For various reasons, native NVDA braille display drivers are preferred over using BRLTTY, so the BRLTTY driver hasn't received much attention recently. I've replied to your points inline:

Replying to DebeeArmstrong:

TeleSensory PowerBraille 40 (PB40) works beautifully under Brltty in Linux. In NVDA, it displays and scrolls fine,

You say here that it scrolls fine, but later, you note that fwinlt and fwinrt don't work. Am I misunderstanding what you mean by "scroll" here?

In most cases, a button was seen by NVDA but not assignned, so I assigned it myself under input gestures.

Most of the BRLTTY commands don't map well to NVDA's commands, which is why there is so much unassigned. It'd be great if you could provide suggestions regarding currently unassigned BRLTTY commands that it would make sense to assign by default.

The left and right buttons, fwinlt and fwinrt respectively, were listed in the tree view for Braille input gestures for scrolling Braille back and forward. They were also recognized when attempting to assign them with the Add button. However they never performed any assigned function and were not seen by input help.

They're not reported by input help because if they were, you wouldn't be able to scroll within a long input help message in order to read it. They should, however, work in an editable text field or the like; they certainly did when I last tested with my display. Can you try assigning them to something ridiculous like report time? That way, we can work out whether these are truly being dropped for some reason.

The large left rocker bar, when pressed up and down without pressing any other buttons, was never recognized by NVDA though Brltty in Linux and other screen readers see it , and its function can easily be reassigned with other screen readers.

If NVDA isn't seeing it, BRLTTY almost certainly isn't passing anything to NVDA. This is something you'd need to work with the BRLTTY developers to understand/resolve. When you say other screen readers see it, do you mean other screen readers via BRLTTY or do you mean native screen reader drivers?

[nvaccessAuto]

Comment 3 by DebeeArmstrong on 2014-06-06 05:14 You ask: "Am I misunderstanding what you mean by "scroll" here?" Yes, I wasn't clear. Those Braille display buttons work using Brltty under Linux; same display, same PCS; I just booted grml to verify. But NVDA doesn't see the buttons, and hence can't scroll display left and right. The other buttons not recognized have the same problem; OK under Linux, but unavailable in NVDA. I don't know how to test Brltty under Windows without NVDA. I'll be getting to your other questions tomorrow. If you know how to test Brltty under Windows without NVDA, let me know and if I should be using an earlier or later build of Brltty or Brlapi let me know. I've built it many times under Linux and NetBSD but don't know how to build it under Windows so hope I can find binaries somewhere. And I'm not ignoring your other questions; just feeling too sleepy to tackle them at the moment, but very willing to test.

[nvaccessAuto]

Comment 4 by jteh on 2014-06-06 05:37 The situation with the scroll buttons and the other unrecognised buttons is definitely a bit different, but I'm not quite sure of the specifics yet.

You can test BRLTTY in Windows without NVDA by setting NVDA to no braille (or exiting NVDA) and opening a command console (e.g. type cmd in the Windows Run dialog).

[LeonarddeR]

cc @adriani90

[Adriani90]

@DebeeArmstrong have you tested BRLTTY and BRL API under Windows as suggested by @jcsteh in his comment above? The current BRLTTY Version is 5.6 and the last BRL API Version is 0.6.7. You can download BRLTTY binaries for Windows and last Version of BRLAPI here: https://brltty.app/download.html#current

The documentation is quite good actually. For an example how it works together with a Screen Reader, see e.g. the chapter "sharing a Braille Display with a Screen Reader". There is an example of how Jaws works in this construct.

The last changes in BRLTTY and BRLAPI can be found here: https://github.com/brltty/brltty-website/blob/master/home/doc/ChangeLog.txt

Basically, BRLTTY is a universal toolkit containing most Braille Display Drivers and text / contracted tables. It is actually something like Liblouis + Drivers. And additionally, it is something like the Bridge between the screenreader, the braille Display driver and the Braille Display itself. So any hardware developer can write a Driver for BRLTTY which is bein included in there. Additionally, every Software developer can learn his or her application to communicate with the BRLAPI and the Display Driver. I did not compare if all Braille tables are included in BRLTTY as well but I guess BRLTTY is in General more extensive than Liblouis. I think Liblouis has much more tables though.

BRLTTY is commonly used by almost all Braille Display Producers.

BRLAPI is actually a Driver for BRLTTY, something like a helper or a control Client which allows Software developers to send Output to Braille in a specific manner. A documentation on BRLAPI can be found here: https://www.systutorials.com/docs/linux/packages/brlapi-0.6.0/Manual-BrlAPI/English/BrlAPI-2.html Well, this documentation is more related to linux but it gives a good overview of how BRLAPI works. @jcsteh what are actually the reasons why NVDA prefers own Braille Display Drivers over BRLTTY Drivers?

cc: @bramd, @michaelDCurran

[LeonarddeR]

As we can't seem to reach the original author, closing this for now to clear up our backlog a bit. If this is still an issue to someone, feel free to reach out and we can reopen/reassess priority, though it won't be higher than a p4 or even a won't fix.