search

microsoft / vscode

Visual Studio Code
https://code.visualstudio.com
MIT License
162.54k stars 28.65k forks source link

[Accessibility] Reading is no longer possible when hovering the mouse over items in the tab and explorer areas of the editor. #224704

Open ryusei-48 opened 1 month ago

ryusei-48 commented 1 month ago

Type: Bug

I use a screen reader. I often use the mouse cursor to read the text below it. However, when I upgraded to version 1.92, I encountered the title problem. NVDA's mouse cursor reading function is used. I hope it will be corrected.

バージョン: 1.92.0 (user setup) コミット: b1c0a14de1414fcdaa400695b4db1c0799bc3124 日付: 2024-07-31T23:26:45.634Z Electron: 30.1.2 ElectronBuildId: 9870757 Chromium: 124.0.6367.243 Node.js: 20.14.0 V8: 12.4.254.20-electron.0 OS: Windows_NT x64 10.0.22631

VS Code version: Code 1.92.0 (b1c0a14de1414fcdaa400695b4db1c0799bc3124, 2024-07-31T23:26:45.634Z) OS version: Windows_NT x64 10.0.22631 Modes:

System Info |Item|Value| |---|---| |CPUs|AMD Ryzen 9 5900X 12-Core Processor (24 x 3700)| |GPU Status|2d_canvas: enabled
canvas_oop_rasterization: enabled_on
direct_rendering_display_compositor: disabled_off_ok
gpu_compositing: enabled
multiple_raster_threads: enabled_on
opengl: enabled_on
rasterization: enabled
raw_draw: disabled_off_ok
skia_graphite: disabled_off
video_decode: enabled
video_encode: enabled
vulkan: disabled_off
webgl: enabled
webgl2: enabled
webgpu: enabled
webnn: disabled_off| |Load (avg)|undefined| |Memory (System)|31.91GB (16.77GB free)| |Process Argv|--crash-reporter-id 819c0f14-6763-4f86-8b72-bd2914389149| |Screen Reader|yes| |VM|40%|
Extensions (54) Extension|Author (truncated)|Version ---|---|--- rust-bundle|1Yi|1.0.0 Bookmarks|ale|13.5.0 vscode-sqlite|ale|0.14.1 vscode-django|bat|1.15.0 vscode-intelephense-client|bme|1.10.4 htmltagwrap|bra|1.0.0 vscode-tailwindcss|bra|0.12.5 vscode-copy-filename|bra|0.1.1 python-environment-manager|don|1.2.4 python-extension-pack|don|1.7.0 rust-syntax|dus|0.6.1 LogFileHighlighter|emi|3.3.0 auto-rename-tag|for|0.1.10 vscode-edit-csv|jan|0.9.2 vsc-python-indent|Kev|1.18.0 iconbuddy-vs-code-plugin|mdd|0.0.3 zenkaku|mos|0.0.3 vscode-language-pack-ja|MS-|1.92.2024073109 csharp|ms-|2.39.29 vscode-dotnet-runtime|ms-|2.1.1 debugpy|ms-|2024.10.0 python|ms-|2024.12.1 vscode-pylance|ms-|2024.7.1 remote-wsl|ms-|0.88.2 cmake-tools|ms-|1.18.44 cpptools|ms-|1.21.5 cpptools-extension-pack|ms-|1.3.0 copy-file-name|nem|1.2.0 innerhtml|nic|0.2.0 autodocstring|njp|0.6.1 indent-rainbow|ode|8.3.1 laravel-blade|one|1.36.1 laravel5-snippets|one|1.18.0 phind|phi|0.25.3 material-icon-theme|PKi|5.8.0 vscode-template-literal-editor|pli|0.10.0 vscode-css-peek|pra|4.4.1 prisma|Pri|5.17.0 inline-sql-syntax|quf|2.16.0 vscode-xml|red|0.27.1 rust-analyzer|rus|0.3.2053 crates|ser|0.6.7 trailing-spaces|sha|0.4.1 vscode-scss-formatter|sib|3.0.0 code-clip-ring|Sir|0.4.0 rust-pack|Swe|0.3.38 even-better-toml|tam|0.19.2 tauri-vscode|tau|0.2.6 cmake|twx|0.0.17 intellicode-api-usage-examples|Vis|0.2.8 vscodeintellicode|Vis|1.3.1 jinja|who|0.0.8 sysinfo-vscode|wra|2.6.0 scss-to-css|yut|3.2.3 (2 theme extensions excluded)
A/B Experiments ``` vsliv368:30146709 vspor879:30202332 vspor708:30202333 vspor363:30204092 vswsl492:30256859 vscorecescf:30445987 vscod805cf:30301675 binariesv615:30325510 vsaa593cf:30376535 py29gd2263:31024239 c4g48928:30535728 azure-dev_surveyone:30548225 a9j8j154:30646983 962ge761:30959799 pythongtdpath:30769146 pythonnoceb:30805159 asynctok:30898717 pythonregdiag2:30936856 pythonmypyd1:30879173 2e7ec940:31000449 pythontbext0:30879054 dsvsc016:30899300 dsvsc017:30899301 dsvsc018:30899302 cppperfnew:31000557 dsvsc020:30976470 pythonait:31006305 dsvsc021:30996838 da93g388:31013173 pythoncenvpt:31062603 a69g1124:31058053 dvdeprecation:31068756 dwnewjupyter:31046869 impr_priority:31102340 nativerepl2:31104044 refactort:31108082 ccplc:31103425 pythonrstrctxt:31103193 wkspc-onlycs-c:31106320 wkspc-ranged-t:31107835 ```
meganrogge commented 1 month ago

I am not familiar with using the mouse cursor functionality. Could you pls provide exact steps and what happened before vs what's happening now (or not)?

cc @rperez030

Thank you

rperez030 commented 1 month ago

NVDA has a feature to read the object under the cursor. That is also part of magnification tools such as zoomtext. I understand they rely on the UIA_BoundingRectangle property to determine the object under the mouse, which I've heard is not very reliable in browsers. I'm personally not very familiar with it since it is not something i would use myself.

ryusei-48 commented 1 month ago

I use the Japanese version of NVDA, so I don't know exactly how to enable that feature in the English version, but it is probably almost the same.

To enable it, open the menu by pressing "NVDA Ki~ + N", select "Settings", and check "Mouse" under the category "Mouse" -> "Report text at mouse cursor position".

Then, in various applications and browsers, when you move the mouse over the text, it will be read out loud.

Because I have low vision, I work with the mouse cursor there and have the text read to me, while somehow perceiving the position of the text.

In one previous version of VS Code (1.92 or earlier), I was able to read out the text without any problem.

Is there some change in the structure of the DOM or is it a problem with the version of the Chrome Driver that is installed?

I have tried various DOM change operations on my end using VS Code's developer tools, but so far it seems to have no effect.

If that didn't help, you could listen for f-focus events on the JavaScript side, write the target text to the element with the "aria-live" attribute, and notify the NVDA side of the text.

ryusei-48 commented 1 month ago

There is one sure cause for not reading them. We need to be sure that this is the case, but we would like to share our findings.

If you want NVDA to read text under the mouse cursor, the object must be directly touching the mouse cursor.

For example, as an example of how to make it unreadable, if you place a transparent DOM over the object for some purpose (using position:absolute, etc.) or create an object using CSS pseudo-classes (before, after, etc.), NVDA will not read NVDA will not read it.

Also, I just downgraded the VS Code version to 1.91. I am reading the text under the mouse cursor with no problem.

meganrogge commented 1 month ago

@deepak1556 know of any change that could've caused this?

deepak1556 commented 1 month ago

@ryusei-48 given this seems like a possible regression, it would help to know which change caused it. We maintain a node.js based CLI tool vscode-bisect that automatically downloads and runs previous VS Code insiders and asks for each build if the issue reproduces. It takes up to 8 steps to find exactly the build that caused the regression.

Steps:

Thanks!

ryusei-48 commented 1 month ago

The range of problematic commits is now as follows

git bisect start && git bisect bad 2fe05272052cf1f0ab2133f7f4874e9a748e8e74 && git bisect good b23e791eb5afbd95f05aa24da7693ce89344a079

Since the problem was with a release build, we ran the command as follows

npx --yes --releasedOnly @vscode/vscode-bisect@latest

The GitHub page summarizing the scope of the problematic commits is below. https://github.com/microsoft/vscode/compare/b23e791eb5afbd95f05aa24da7693ce89344a079...2fe05272052cf1f0ab2133f7f4874e9a748e8e74

Thanks!

ryusei-48 commented 1 month ago

We first reported that the text in the Explorer area and editor tabs was no longer being read out loud, but after much verification, we found that the text under the mouse cursor was no longer being read out loud over a fairly wide area.

For example, the popup that appears in the lower right corner of the screen when there is some notification, the description popup that appears when the mouse cursor is over a TypeScript keyword or type, and all items on the settings page.

It seems to me that something is wrong with the entire VS Code screen.

hwf1324 commented 1 month ago

I'm having the same issue.

I am using the https://github.com/hwf1324/objectViewer NVDA add-on I developed to view the layout of the NVDAObject.

Here is an example:

屏幕截图 2024-08-07 162921

The Report Issues window for the latest preview version

屏幕截图 2024-08-07 163245

Report issue window of an earlier preview version

The difference between them can be observed with a new NVDAObject. Perhaps as mentioned above there is something blocking the mouse from finding the object below?

hwf1324 commented 1 month ago

On a side note, could this be related to Electron? Because one of the apps I use had a similar problem after one of the recent updates. It uses Electron as well.

Of course I haven't investigated or tested it, just making a hypothesis.

deepak1556 commented 1 month ago

Thanks @ryusei-48 for the bisect results, the commit does reference an Electron update we had in 1.92 it could be very well be a regression in Chromium.

I don't have the setup to confirm this issue yet, If anyone is interested in the meantime to investigate further please try the following next steps to bisect chromium builds,

python tools/bisect-builds.py -a win -g 122.0.6261.156 -b 124.0.6367.243 --use-local-cache --verify-range -- --no-first-run https://vscode.dev
ryusei-48 commented 1 month ago

Thank you for your research. I just did a bisect of cromium.

As a result, it seems that there is a problem with cromium as you mentioned.

The problem is happening in the following commit ranges f777d3b07c236181ee6f8899523171e871d7abe9 -> 28ea064cc7c474a6e850b53e35487ad6bdab58d0

CHANGELOG URL: https://chromium.googlesource.com/chromium/src/+log/f777d3b07c236181ee6f8899523171e871d7abe9..28ea064cc7c474a6e850b53e35487ad6bdab58d0

deepak1556 commented 1 month ago

Thanks for the quick turnaround, but looking at the commit history in that range only one commit is relevant to a11y but that is not bundled as part of our application. It could be that the bisect range is incorrect, can you try the chromium bisect again to confirm.

ryusei-48 commented 1 month ago

OK, I tried again. The results were a little different.

PS> python bisect-builds.py -a win -g 122.0.6261.156 -b 124.0.6367.243 --use-local-cache --verify-range -- --no-first-run https://vscode.dev
Downloading list of known revisions. If the range is large, this can take several minutes...

Loaded revisions 1625-1338356 from C:\Users\yamaz\Downloads\test\.bisect-builds-cache.json
Downloading revision 1250580...
Received 209320771 of 209320771 bytes, 100.00%
Trying revision 1250580...
Revision 1250580 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1274506...
Trying revision 1274506...
Revision 1274506 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: b
Downloading revision 1262540...
You have about 10 more steps left.
Bisecting range [1250580 (good), 1274506 (bad)].
Trying revision 1262540...
Revision 1262540 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1268390...
You have about 9 more steps left.
Bisecting range [1262540 (good), 1274506 (bad)].
Trying revision 1268390...
Revision 1268390 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1271471...
You have about 8 more steps left.
Bisecting range [1268390 (good), 1274506 (bad)].
Trying revision 1271471...
Revision 1271471 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: b
Downloading revision 1270530...
You have about 7 more steps left.
Bisecting range [1268390 (good), 1271471 (bad)].
Trying revision 1270530...
Revision 1270530 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1270722...
You have about 6 more steps left.
Bisecting range [1270530 (good), 1271471 (bad)].
Trying revision 1270722...
Revision 1270722 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1270902...
You have about 5 more steps left.
Bisecting range [1270722 (good), 1271471 (bad)].
Trying revision 1270902...
Revision 1270902 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1271272...
You have about 4 more steps left.
Bisecting range [1270902 (good), 1271471 (bad)].
Trying revision 1271272...
Revision 1271272 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1271386...
You have about 3 more steps left.
Bisecting range [1271272 (good), 1271471 (bad)].
Trying revision 1271386...
Revision 1271386 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1271438...
You have about 2 more steps left.
Bisecting range [1271386 (good), 1271471 (bad)].
Trying revision 1271438...
Revision 1271438 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
Downloading revision 1271458...
You have about 1 more steps left.
Bisecting range [1271438 (good), 1271471 (bad)].
Trying revision 1271458...
Revision 1271458 is [(g)ood/(b)ad/(r)etry/(u)nknown/(s)tdout/(q)uit]: g
You are probably looking for a change made after 1271458 (known good), but no later than 1271471 (first known bad).
CHANGELOG URL:
  https://chromium.googlesource.com/chromium/src/+log/c84e8a928888a874d39e0f3ef70b13721b8bdbdf..ab56da8dd8d41d786caf4954fd8e9aea55beebb0
hwf1324 commented 1 month ago

Related: https://github.com/electron/electron/issues/42945

hwf1324 commented 1 week ago

I created an NVDA add-on to temporarily fix this.

https://github.com/hwf1324/mouseEnhancements