Improve braille support in spreadsheets and tables

[LeonarddeR]

In the current situation, braille support in Spreadsheet applications (such as Excel) is very limited. Most notably, it is not possible to view a complete row on a braille display, only single cells. Same applies to tables in Word.

This could be solved in multiple ways, one of which could be adding a table row review mode so both braille and speech users can benefit from this. Of course, row review mode is very obnoxious in case of huge rows.

[feerrenrut]

I think this requires some more discussion to properly define the requirements, and explore some of the possibilities. However that said, this is a good opportunity for innovation, and could potentially improve workflows significantly.

Setting this to a priority 3.

Any further thoughts, or examples on how this could work would be appreciated.

[jcsteh]

@leonardder, a good starting point might be how other screen readers handle this if you have that info. I'm particularly interested in Excel, as rows are extremely long in Excel. We don't have to copy others, of course, but learning from history is also not a bad thing. :)

[LeonarddeR]

It seems that somehow, Excel support in both SuperNova and JAWS is majorly broken at the moment, probably due to a bug in Excel. I'm afraid I can't get a working environment to make a report on this yet.

[dkager]

Our national library for the blind used to use "semicolon tables" in their productions. That is: col1; col2; col3; col4

They have since switched to Word tables, but users still request the old way sometimes. This appears to be very similar to how NVDA brailles list items. The advantage is that it doesn't take up as much space as the purely physical representation that SuperNova uses. But if the column contains a semicolon that would have to be dealt with somehow. You also don't have any spatial information, e.g. if something is cropped or if there is a lot of whitespace.

[LeonarddeR]

Turns out that I never came back to the report on how other screen readers do this.

As for Dolphin, things are still majorly broken, so there's nothing interesting we can learn from them.

JAWS has the possibility to print rows and columns on a braille display. That is, you can change braille view between single cell, current row and current column. They print around 10 cells on every cursor move and have ⣿ as separator between the cells.

I've played a bit with the object model in Excel, and I can retrieve the text of around 10 cells at once without a noticeable delay. Printing the text of 100 cells takes around half a second. However, it is probably unnecessary and undesirable to do the latter.

One question we should answer first is: should the possibility to view multiple cells at once be limited to braille, or is there the need for a mode where you can read multiple cells at once with speech? As I think the latter gives way too much information at once, I guess there's only desire for braille spesific functionality on this subject.

It will also require some discussion to come up with a good implementation proposal. E.g. some tables have separate objects (excel, the web, Java), whereas others do not (tables in word). I assume the simplest way to come with a common denominator is to create a new type of braille region which creates braille output based on a range of cells and their contents.

[LeonarddeR]

@michaeldcurran: Is it true that there is not yet proper UIA support for table cells in Excel? How is this supposed to work under Windows 10s?

[michaelDCurran]

This is correct. Future work needs to be done. NV Access will be working with the MS Office team shortly to address this issue.

However, it does not technically affect Windows10 S. It is true we cannot inject into Office products on 10 S, but our Excel support does not rely on injection, only the object model out-of-process. However, if MS Office was ever to be run in Windows Defender Application Guard, then we most certainly would need to fall back to UIA as the object model would be inaccessible.

Note that MS word right now is inaccessible on windows10 S as that does rely on injection. Thus why we are moving fast towards UIA access for MS Word.

[Adriani90]

I get requests from many users, especially from previous Cobra screen reader users, who ask such a feature. Especially for MS Excel it would be nice to have. cc: @Qchristensen

[Qchristensen]

Yes, we also get requests from users wanting the ability to read either a row, a column, or to use "Say all" on a spreadsheet to read a whole sheet at once. Both in Braille and speech.

Currently, a workaround for this is to select the row, column, or whole sheet, copy it and either read the contents of the clipboard or paste into Notepad to read.