vsCode statusbar not properly read

[isidorn]

1. Open VS Code insiders
2. Trigger read status bar in NVDA command

VS Code issue https://github.com/microsoft/vscode/issues/96392

Comments from @joanmarie on what Orca is doing in Linux

Orca is doing an accessibility tree dive from the bottom up, looking for something with the accessible role of "status bar". I recently had to change it to sanity-check candidate status bars for the "showing" and "visible" accessibility states because VSCode has -- or at least used to have -- some offscreen element with ARIA role status and no contents. Orca was finding it first and not bothering to look any further.

@feerrenrut: Yeah, I ran into that Chrome hit-test thing myself in a different context. Orca now asks twice -- at least for the place where the issue came up.

[feerrenrut]

I think this issue should be specific to vsCode. The browser case is a little more tricky since browsers may have their own window chrome status bar in addition to any HTML defined status bar, though I'm happy for that to be split into a separate bug. It may be that for "browser based" applications that don't have a status bar (most these days) we use appModules to opt-in to retrieving the status bar from the content.

See PR for overriding status bar with app modules: https://github.com/nvaccess/nvda/pull/9792

[Adriani90]

Cc: @josephsl, @JulienCochuyt is this a candidate for a custom status bar? Or is this not possible in web apps?

[josephsl]

Hi,

What Julian wrote in #9792 deals with retrieving status bar objects for the focused app, not necessarily for web applications. This is because web apps are hosted inside a larger user interface window, so coming up with custom status bar fetcher will require looking at either the web app or the web browser window.

A few years ago I, @JCSTeh, and one or two others had a debate as to which class is responsible for telling NVDA to look for specific status bar objects. Some said overlay classes should do it, as they have an advantage in knowing the context surrounding where they are (often, overlay classes are chosen based on element properties such as name, window class name, UIA class name, role, or sometimes even foreground object or what the object's neighbor is). On the other hand, some said app modules should be the ones NVDA should talk to, as they come in as long as an app is in use and are best positioned to describe what the screen shows.

Web applications pose certain issues. in addition to the typical layout I described, they are composed of many elements, some of them HTML-based, others enhanced with ARIA, some exposing IA2 information while some may not, some only appearing when situations demand it, and other possibilities. To describe a status bar for a web application is akin to asking NVDA (or for that matter, content rendering engines, screen readers, and others) to look for a specific tree inside a forest. And since web applications are hosted inside a browser window, NVDA will wonder as to which forest to visit: the web application, web elements surrounding the app in question, or browser interface.

IN the ideal world, a heuristic approach that combines best of both worlds (overlay class (contextual) or app module (screen-based)) might be a "good" compromise. I say good in quotes because of an important factor that's been under discussion lately: performance and responsiveness. If the web element in question knows an efficient way to fetch a status bar object, performance bottleneck comes down to instantiating an NVDA object for it (or getting one). However, unless a web application is well designed, status objects may occur anywhere, and given our subjective nature, developers can claim something is a status bar object when it is actually not. Adding to the problem is that, to NVDA, there are many "cars" to choose from - hundreds of web elements to choose from, many of them showcasing similar properties. Failing that, an app module for a web browser can claim that it knows the fastest way to locate status bars, which may not be the case if it turns out it is not the element you had in mind. Only then screen position is used, which is known to fail under some cases (File Explorer is a good example of this).

Another thing to consider: pace of change is accelerating. Although many people may say rate of change is constant, it is not. In other words, what NvDA considered a status bar for a web app today may not be so a week from now, or an app that didn't expose status bar objects correctly can do so in a matter of hours to days, making NVDA's workaround unnecessary. This is one of the reasons why I did say an ideal heuristic may provide a "good " compromise - the caveat here is ability for everyone to ride the same bus, which actually doesn't happen a lot in practice (Windows 10 is a good example).

To answer Adriani's question, #9792 does not apply cleanly to web applications at this time. But at least we have something to think about, and hopefully a future iteration may provide a way to let NVDA take care of a situation like this.

Thanks.

[feerrenrut]

@josephsl you may have missed that this issue is about vsCode. Which we can easily have an appModule for with it's own strategy for getting the status bar. The two candidates strategies for fixing the current issue (caused by inaccurate results hit-testing in chromium) are:

• When status bar isn't found, wait and try again.
  • Chromium needs to re-render and cache the area to get accurate hit test information.
• As Orca does, search the accessibility tree from the bottom up, looking for something with the accessible role of "status bar".

@isidorn I have removed the following browser based steps to reproduce from the issue description to try to keep this on track. To fix this for general web apps running in any browser will require a number of different considerations. Hope you don't mind.

Removed from initial description

Alternative repro steps for a general web page: 1. Have a web page with a `role=status` element 2. Open that web page in chrome 3. Trigger the NVDA command to read status bar -> "No status line found" is read 🐛

I'm happy for anyone to turn this into a new issue, where the comment from @josephsl is very relevant. @josephsl I'll hide your comment for now to prevent de-railing this issue.

[josephsl]

Hi,

Ah, makes more sense. In this case, I think @LeonardDer is in the best position to take a look at this more thoroughly (I don't use VS Code at this time).

Thanks.

[isidorn]

@feerrenrut I do not mind. Thanks for looking into this!

[isidorn]

Friendly ping regarding this issue. Are there any updates or plans to tackle this one?

[feerrenrut]

Sorry @isidorn we haven't been able to prioritize this yet.

[Adriani90]

@cary-rowen, @mltony, @LeonarddeR is this issue still existing with NvDA 2024.3 beta?

[Adriani90]

We didn't hear any updates from users who could reproduce it in the past upon request since more than 3 years. I am closing as abandoned. If you are still having this issue, please comment and we can reopen.

[cary-rowen]

I have to say, if you have tried the latest VS Code you will easily reproduce, the issue with NVDA not reporting the status bar.

[Adriani90]

Thanks for the update.

[hwf1324]

I have to say, if you have tried the latest VS Code you will easily reproduce, the issue with NVDA not reporting the status bar.

Please note that this is a different issue than #17339.

https://github.com/microsoft/vscode/issues/96392#issuecomment-620651967

However, there is no doubt that this issue still exists.

Even after using the fix add-on, the status bar is only reported correctly after the first reading of the status bar.