Allow NVDA to control multiple braille displays at the same time

[LeonarddeR]

Although the title of this issue might sound strange (why should one ever want to use multiple braille displays at once?), I think there are several good and valid reasons why it would be a great feature to have in NVDA.

1. In the current state of NVDA Remote, when a master controls a slave, displays are mirrored. Whenever the master scrolls, the slave display scrolls automatically and vice versa. NVDA Remote limits the display size to the size of the smallest display in a session. When every display is controlled separately (i.e. could have its own display size and buffers), people will always be able to use the full size of their display, even when controlling a system that has a smaller display attached. Also, when one user scrolls the display, this won't interfere with the reading of the other user.

2. It is a very helpful feature in case of training situations. When I train people on behalf of @babbagecom, I often use a separate keyboard. However, to see what the trainee is seeing, I can only rely on speech. Same applies the other way around. When there is multi braille display support, it is simply possible to attach two displays to the machine, so that a trainer and student can have their own braille output.

3. It would be possible to support chaining braille displays together, for example use two 40 cell displays in order to simulate a 80 cells one.

4. Alternatively, you could show one window on the first display and another on the second, show local braille output on the first and remote braille output on the second, etc.

5. According to a discussion at sight city 2018, there are some users with very specific demands that require two separate displays being connected to a system. For example, one might want to use one display to control the local system, and the other display to quickly read a telephone number of an incoming call. The only screen reader that has been able to deliver this functionality is Baum Cobra, however, due to the currently uncertain situation of Baum, people could no longer rely on Cobra for this functionality.

[bramd]

I think that it might also ease switchign displays. E.g. a user works at the office with a laptop on a USB display. He puts the laptop into sleep and unplugs everything. Later, while traveling, he wants to continue where he left off and use the same laptop with a Bluetooth display.

If NVDA supports one display at a time, it has to figure out that the previous one has been disconnected and the Bluetooth one is connected. Figuring out the disconnect might take some time. If it supports multiple active displays at once, the user wouldn't have to wait for a timeout to occur and can immediately start using the other display. In the background the old display driver might still run and get terminated after a connection timeout, but that wouldn't bother the user.

I think this use case also implies that there may be multiple active instances of the same braille driver at a time. If the Bluetooth display is powered by the same driver as the USB one for example.

[feerrenrut]

I can see the value in looking into this, however I think it could be quite a big project. I would be interested to see something hacked together as a prototype to explore what issues we may run into. For instance running multiple copies of the driver as mentioned by @bramd. I will set this to a priority 3.

[Adriani90]

@leonardder should I copy my thoughts in this issue as well?

[LeonarddeR]

@Adriani90 commented on 22 mrt. 2018 15:35 CET:

@leonardder should I copy my thoughts in this issue as well?

Sorry, I missed this message. Go ahead! I will also update the issue description to reflect actual ideas and findings.

[Adriani90]

Ok, following there are some examples of Output on second Braille Display and some other use cases. @leonardder, if you include them in the issue description, then I can delete my following comment to avoid duplicate Information.

[Adriani90]

Output on second braille display:

Use case 1: working with SAP and a phone Software at the same time or when editing multiple tables in SAP:

• Copy / show current screen with empty lines. • Copy / show current screen without empty lines. • Copy / show current data of win.-clipboard. • Copy / show selected data. • Copy / show last 10 speechmessages, with the last at the top (used a lot in cells of tables and grids in SAP. The header is not saved in the buffer) • Copy / show last 10 brailleoutputs (the same as with speech).

Regarding phone Software:

• Show vertically ordered Information of the pfone Software window in a horizontal way on the first Braille Display (i.e. Long term customer, phone number, Name, availability etc.). This is especially the case in MyPBX softphone. • Show i.e. the SAP customer database on second Braille Display and allow selecting entries, while displaying current phone call Information on the first Braille Display.

[Adriani90]

This commands could be pressed on the second Braille Display: • Set begin of selection • Set end of selection and copy to clipboard (possible to select text from the 2nd display and paste it where needed) • Go to top of display2. • Go to end of display2. • Search text forwards and backwards on display2. • Jump and speak the previous/next sentence on display 2 (very useful for big displays with 60 cells and more) • Speak sentences on second display in a different language than the main NVDA instance is set on (very useful for translators). • Save input on braille display 2 in a buffer, so that it can be restored after restarting the screen reader (very useful when chrashes occur).

[Adriani90]

If we are thinking further, another use case would be the Bloomberg terminal, where we could think about connecting as many Braille displays to the terminal as connected Screens. On the Bloomberg terminal, up to 6 Screens can be connected. Every Screen can be accesed with the Bloomberg Keyboard, so it is possible to navigate through Information on Screen 1, Change and navigate through information on Screen two etc. Same Scenario could apply to connected Braille Displays. For example: • display Country Information on Screen 1 and Braille Display 1 • Display sector Information on Screen 2 and BD2 • Display Company Information on Screen 3 and BD3 • Display the launchpad on Screen 4 and BD4 (BD4 could be set as main Braille Display)

The launchpad is a separate window where the screens can be reorganized or where some General Information can be shown. From the launchpad one can assign a certain type of Information to a certain Screen.

[Qchristensen]

I had a request for this from a user (Valeriano Septién) via email, whose use case is: "The idea of having several braille displays is for teaching braille reading to a small group of people. Teaching with paper books, has the disadvantage that the points are crushed with use and it is more difficult to appreciate them with touch. If we can connect several braille lines to a computer, we can follow all (4 or 5 people) the same text with the same point quality."

[josephsl]

CC @LeonardDer

[XLTechie]

This seems like something that should be done through an add-on. It is very unusual functionality.

One question: are all the braille displays the same? All different? Some the same and some different?

[josephsl]

Hi, although some are same (similar protocol, connectivity setup, etc.), most are different. For example, even though they are really one lineage, BrailleNote Apex and Braille Note Touch Plus are two completely different braille displays (I can testify about this because I happen to use both). Thanks.

[XLTechie]

Sorry @josephsl, I was directing that specifically to the use case @Qchristensen was asking about. I was unaware of the deep history on this question.

My thinking was that it would be easier to do if a different driver was in use for each display, rather than trying to effectively multiplex one driver for multiple displays.

[josephsl]

Hi, one way is using threads and listeners so each display can output the same information based on what NVDA wants it done. The obvious problem arises when users try to use input keys on different displays at the same time. There might be other ways of accomplishing this, many of them involving facilities found in Python 3 including asyncio. Thanks.

[Adriani90]

@XLTechie I am afraid but this is not unusual functionality. There are hundreeds of users of Cobra screenreader which is discontinued and they all worked with multiple Braille Displays before. Especially in corporate Environments there are plenty of use cases and the potential of such a feature is really big, also because this would allow to speak to Multi line Displays like canute or grafiti. However, I think implementing such a feature will require huge work and Money and can possibly be acomplished only if a bigger Institution decides to invest time and Money in this regard.