search

nvaccess / nvda

NVDA, the free and open source Screen Reader for Microsoft Windows
https://www.nvaccess.org/
Other
2.1k stars 634 forks source link

Unable to use Visual Studio with NVDA: NVDA continuously keeps on reading whole window content #14086

Closed akash07k closed 2 years ago

akash07k commented 2 years ago

Steps to reproduce:

  1. Update to latest NVDA Alpha
  2. Open Visual Studio 2022 or Visual Studio Installer.
  3. Observe.
  4. Press tab or any other navigation key.
  5. Observe

    Actual behavior:

    NVDA continuously keeps on announcing whole window content inside Visual Studio and we are unable to use it since the announcement doesn't get interrupt even if we manually navigate. seems that this issue is started appearing after merging these PRs:

    14067

    14080

    Speech output below:

    
Microsoft Visual Studio Community 2022
Microsoft Visual Studio Int Preview  window
Microsoft Visual Studio Int Preview 
Minimize 
Restore Down 
Close 
blank
Visual Studio2022 
Visual Studio 
2022 
Recent projects 
Open recent 
Open _recent 
blank
blank
blank
Open recent 
Get started 
blank
Start learning 
blank
Start learning 
Learn the basics of Visual Studio by making an ASP.NET Core app 
Clone a repository 
blank
_Clone a repository 
Get code from an online repository like GitHub or Azure DevOps 
Open a project or solution 
blank
Open a _project or solution 
Open a local Visual Studio project or .sln file 
Open a local folder 
blank
Open a local _folder 
Navigate and edit code within any folder 
Create a new project 
blank
Create a _new project 
Choose a project template with code scaffolding to get started 
blank
blank
blank
blank
blank
blank
blank
Continue without code 
Continue without code 
Continue _without code 
Continue _without code 
Continue without code 
blank
Microsoft Visual Studio Int Preview 
Microsoft Visual Studio Int Preview 
Recent projects 
_Search recent (Alt+S) 
_Search recent (Alt+S) 
_Search recent (Alt+S) 
_Search recent (Alt+S) 
Search 
Drop Down 
Search Control 
Search Control 
Today 
ConsoleApp1.sln 
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
blank
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
ConsoleApp1.sln C:\Users\v-akkak\source\repos\ConsoleApp1 
Search Control  grouping  collapsed
_Search recent (Alt+S)  edit  blank
No supported apps in use for mic mute
Open recent  list
Today  grouping  expanded  1 of 1
Start learning  button
Clone a repository  button  C
Open a project or solution  button  P
Open a local folder  button  F
Create a new project  button  N
blank
blank
Create a new project 
blank
_Recent project templates 
_Recent project templates 
_Recent project templates 
blank
blank
_Recent project templates 
blank
blank
LanguageFilter 
PlatformFilter 
ProjectTypeFilter 
Project Templates 
_Back 
_Next 
Search Control 
_Search for templates (Alt+S) 
_Search for templates (Alt+S) 
_Search for templates (Alt+S) 
_Search for templates (Alt+S) 
Search 
Drop Down 
Search Control 
Search Control 
Microsoft Visual Studio Int Preview  window
JavaScript Express Application 
blank
JavaScript Express Application 
A basic JavaScript Express application template which is bootstrapped by running your global install of npx and npm. 
blank
JavaScript 
blank
Web 
JavaScript Express Application 
JavaScript Express Application 
Echo Bot (Bot Framework v4 - .NET Core 3.1) 
blank
Echo Bot (Bot Framework v4 - .NET Core 3.1) 
Echo Bot Template for Bot Framework v4.  A good template if you want a little more than "Hello World!".  Echo Bot simply "echoes" back to the user anything the user says to the bot. 
blank
C# 
blank
Windows 
blank
macOS 
blank
Linux 
blank
AI Bots 
Echo Bot (Bot Framework v4 - .NET Core 3.1) 
Echo Bot (Bot Framework v4 - .NET Core 3.1) 
Core Bot (Bot Framework v4 - .NET Core 3.1) 
blank
Core Bot (Bot Framework v4 - .NET Core 3.1) 
Core Bot Template for Bot Framework v4.  Our most feature rich template, it shows how to use LUIS and multi-turn conversational patterns. 
blank
C# 
blank
Windows 
blank
macOS 
blank
Linux 
blank
AI Bots 
Core Bot (Bot Framework v4 - .NET Core 3.1) 
Core Bot (Bot Framework v4 - .NET Core 3.1) 
Core Bot with Tests (Bot Framework v4 - .NET 6.0) 
blank
Core Bot with Tests (Bot Framework v4 - .NET 6.0) 
Core Bot Template with Unit Tests for Bot Framework v4.  Same features as Core Bot, plus a full unit test project. 
blank
C# 
blank
Windows 
blank
macOS 
blank
Linux 
blank
AI Bots 
Core Bot with Tests (Bot Framework v4 - .NET 6.0) 
Core Bot with Tests (Bot Framework v4 - .NET 6.0) 
Empty Bot (Bot Framework v4 - .NET Core 3.1) 
blank
Empty Bot (Bot Framework v4 - .NET Core 3.1) 
Empty Bot Template for Bot Framework v4.  A good template if you want a skeleton project or want to take sample code from the documentation and paste it into a minimal bot in order to learn. 
blank
C# 
blank
Windows 
blank
macOS 
blank
Linux 
blank
AI Bots 
Empty Bot (Bot Framework v4 - .NET Core 3.1) 
Empty Bot (Bot Framework v4 - .NET Core 3.1) 
Android Wear Application 
blank
Android Wear Application 
A project for creating a .NET Android Wear application 
blank
C# 
blank
Android 
blank
Mobile 
Android Wear Application 
Android Wear Application 
Console App 
blank
Console App 
A project for creating a command-line application that can run on .NET on Windows, Linux and macOS 
blank
C# 
blank
Linux 
blank
macOS 
blank
Windows 
blank
Console 
Console App 
Console App is unpinned 
blank
blank
blank
blank
C# 
blank
blank
ASP.NET Core Web App 
blank
ASP.NET Core Web App 
A project template for creating an ASP.NET Core application with example ASP.NET Razor Pages content. 
blank
C# 
blank
Linux 
blank
macOS 
blank
Windows 
blank
Cloud 
blank
Service 
blank
Web 
ASP.NET Core Web App 
OBS 27.2.4 (64-bit, windows) - Portable Mode - Profile: Untitled - Scenes: Untitled  window
Controls  window
Start Recording  check box  not checked
Issues · nvaccess/nvda - Brave  row 1  column 2  2 of 3
NVDA Speech Viewer  row 1  column 3  3 of 3

<!--
Use "Speak command keys" (NVDA+4) and speech viewer to copy and paste here. 
Use braille viewer to copy and paste here.
You may additionally include an explanation.
-->

### Expected behavior:
NVDA shouldn't read whole window content automatically
<!--
Use "Speak command keys" (NVDA+4) and speech viewer to copy and paste here. 
Use braille viewer to copy and paste here.
You may additionally include an explanation.
-->

### NVDA logs, crash dumps and other attachments:
Please check attached recording too
### System configuration
#### NVDA installed/portable/running from source:
Installed
#### NVDA version:
NVDA version alpha-26412,28b47560
#### Windows version:
Windows 11 (10.0.25188.0)
#### Name and version of other software in use when reproducing the issue:
Visual Studio 2022
#### Other information about your system:

### Other questions
#### Does the issue still occur after restarting your computer?
Yes
#### Have you tried any other versions of NVDA? If so, please report their behaviors.
It was fine with previous versions
#### If NVDA add-ons are disabled, is your problem still occurring?
Yes
#### Does the issue still occur after you run the COM Registration Fixing Tool in NVDA's tools menu?
Yes

https://user-images.githubusercontent.com/6381747/187363652-0eb482f9-f2ee-496e-9050-19eb96a474c3.mp4
cary-rowen commented 2 years ago

I did not reproduce this.

cary-rowen commented 2 years ago

No, I may have to retract my comment because I had a problem with this in the WPF form designer, NVDA kept reporting "default". and the issue appeared in alpha-26387, Check the following configurations in the NVDA Advanced Settings panel: Registration for UI Automation events and property changes: Automatic (prefer selectively)

Below is the log: nvda-alpha-26387.txt

cary-rowen commented 2 years ago

cc @codeofdusk @josephsl

akash07k commented 2 years ago

Have you checked "Selective UIA" checkbox from advance settings?

On 8/30/2022 6:30 PM, Rowen wrote:

I did not reproduce this.

*

NVDA Version: alpha-26412,28b47560 (2022.4.0.26412)

*

Microsoft Visual Studio Community 2022 Version 17.3.2

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/14086#issuecomment-1231634375, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQWBMZOOQ326OXA7EARXUTV3YAWDANCNFSM6AAAAAAQACCDWM. You are receiving this because you authored the thread.Message ID: @.***>

akash07k commented 2 years ago

Yup

On 8/30/2022 7:06 PM, Rowen wrote:

No, I may have to retract my comment because I had a problem with this in the WPF form designer, NVDA kept reporting "default". and the issue appeared in alpha-26387, Check the following configurations in the NVDA Advanced Settings panel: Registration for UI Automation events and property changes: Automatic (prefer selectively)

Below is the log: nvda-alpha-26387.txt https://github.com/nvaccess/nvda/files/9453253/nvda-alpha-26387.txt

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/14086#issuecomment-1231680369, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQWBM253ZAQUEOK4IRPKMDV3YE5PANCNFSM6AAAAAAQACCDWM. You are receiving this because you authored the thread.Message ID: @.***>

josephsl commented 2 years ago

Hi,

Looks like an event flood to me. Can you try:

  1. Install Event Tracker add-on.
  2. Restart NVDA with debug logging enabled.
  3. Open Visual Studio and repeat the steps noted in the original comment.
  4. Open log viewer, copy log fragments relevant to Visual Studio (this will include Event Tracker output), and paste it as an attachment or an issue comment.

Note that I cannot reproduce this from VS startup screen (the one described in the original comment (I'm using alpha.26412 myself). Since my UIA item status event handler could be involved, I'll keep an eye on this issue.

Thanks.

akash07k commented 2 years ago

@josephsl sure, I'll come back with the results

akash07k commented 2 years ago

I suspect that it only happens if we choose "global" from UI Automation Registration combobox

akash07k commented 2 years ago

Yes, it is an event flud. Happens only if we select "Global"

cary-rowen commented 2 years ago

When i choose "automatically" it will happen

josephsl commented 2 years ago

Hi, note that selective registration is the default in Windows 11 Version 22H2 (build 22621) and later. It may not be enough to create an app module level variable to control this behavior as UIA handler thread must be restarted for changes to take effect. Thanks.

akash07k commented 2 years ago

@josephsl I think it would be better if Microsoft can fix it themselves in Visual Studio itself, but I don't think that they will be willing to take it up. Even better, if this event storm can be fixed in VS itself, then it will improve the performance with screen readers a lot

akash07k commented 2 years ago

@cary-rowen yes, with me too, if I choose automatically, it starts happening.

codeofdusk commented 2 years ago

I think it would be better if Microsoft can fix it themselves in Visual Studio itself

Yes. CC @rperez030.

akash07k commented 2 years ago

@codeofdusk Thanks for CCIng @rperez030 I really beg for fixing this. I'm plagued by this performance hit with VS since many years.

feerrenrut commented 2 years ago

@akash07k can you confirm if there is an acceptable workaround? @josephsl do you have enough information to confirm the cause of this issue? Is this this a regression?

akash07k commented 2 years ago

@feerrenrut As of now, the only workaround to bypass this issue is to select "Selectively" from "Registration for UI Automation events and property changes" However, I feel that this workaround is not that optimul and the issue should be fixed both at NVDA's side as well as Visual Studio should stop the event fludding. If Microsoft fixes the event fludding in Visual Studio, then it will be a best fix for everybody and screen reader side tinkering won't be required.

codeofdusk commented 2 years ago

@feerrenrut As of now, the only workaround to bypass this issue is to select "Selectively" from "Registration for UI Automation events and property changes"

This is default on SV2+, see #14018.

If #14080 is indeed the cause, maybe it could be restricted to SV2+ as well?

akash07k commented 2 years ago

@feerrenrut in my opinion we should approach Microsoft and make then understand that how severe event fludding is. they should take it on priority

akash07k commented 2 years ago

@codeofdusk Actually with me it happens if I choose "Automatic (Selective preferred" option. I think that it still prefers global registration

codeofdusk commented 2 years ago

@akash07k What do the following lines in the NVDA Python console return on your system?

import UIAHandler
UIAHandler.utils._shouldSelectivelyRegister()
seanbudd commented 2 years ago

Has this been reported to Microsoft yet?

codeofdusk commented 2 years ago

Has this been reported to Microsoft yet?

Not to my knowledge (unless @akash07k filed), but I'll see if I can flag it internally.

rperez030 commented 2 years ago

Thanks for looping me in. @codeofdusk have you been able to reproduce the issue. I cannot. I'm running the latest build from the preview channel and manually set selective registration to global. @akash07k would you mind sharing the log showing that flood of events? You can send it to robertoperez@microsoft.com if you don't want to post it here. We are currently investigating what may be a related issue, and that would be very helpful in resolving this.

akash07k commented 2 years ago

Definitely, Can we connect on teams for more in depth discussion on this?

If you agree, then I'll connect with you on Monday in  your timezone.

On 9/10/2022 12:56 AM, Roberto Perez wrote:

Thanks for looping me in. @codeofdusk https://github.com/codeofdusk have you been able to reproduce the issue. I cannot. I'm running the latest build from the preview channel and manually set selective registration to global. @akash07k https://github.com/akash07k would you mind sharing the log showing that flood of events? You can send it to @.*** if you don't want to post it here. We are currently investigating what may be a related issue, and that would be very helpful in resolving this.

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/14086#issuecomment-1242378153, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQWBM2TMWEM2Q4CYJFFZILV5OFM3ANCNFSM6AAAAAAQACCDWM. You are receiving this because you were mentioned.Message ID: @.***>

rperez030 commented 2 years ago

Ok, let's connect on Monday. I am in US eastern time.

akash07k commented 2 years ago

Sure,

Thanks

On 9/10/2022 9:05 AM, Roberto Perez wrote:

Ok, let's connect on Monday. I am in US eastern time.

— Reply to this email directly, view it on GitHub https://github.com/nvaccess/nvda/issues/14086#issuecomment-1242608703, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABQWBM7T2GRME3NR556KZDLV5P6ZTANCNFSM6AAAAAAQACCDWM. You are receiving this because you were mentioned.Message ID: @.***>

michaelDCurran commented 2 years ago

Using NVDA alpha-26524,bc1f92f9 With either selective or global UIA registration in NvDA, I am unable to reproduce this on Windows 11 10.0.25193 and either VS 2022 17.3 (stable) or VS 2022 17.4 (preview). @akash07k and @cary-rowen Can you please report your exact Visual Studio version? Can you also confirm that installing an older version of NvDA and testing the various UIA registration scenarios that you cannot reproduce this? Please confirm that you have done this test after the problem started. I.e. to completely rule out that Visual Studio changed.

cary-rowen commented 2 years ago

Hi, Using NVDA alpha-26524,bc1f92f9 With global UIA registration in NvDA. I can still reproduce, I hear "default" several times for each tab key press in the form designer. Note that I can only reproduce this in a WPF project not WinForm.

cary-rowen commented 2 years ago

Hi, @josephsl I followed your steps with a full log

  1. Install Event Tracker add-on.
  2. Restart NVDA with debug logging enabled.
  3. Open Visual Studio and repeat the steps noted in the original comment.
  4. Open log viewer, copy log fragments relevant to Visual Studio (this will include Event Tracker output), and paste it as an attachment or an issue comment.

Below is the log attachment, it has no privacy and I don't care: nvda_Log_With_EventTracker.log

michaelDCurran commented 2 years ago

I think that @akash07k and @cary-rowen might be talking about different things. @cary-rowen is talking about hearing the word "default" a lot in designer. Perhaps this was due to NVDA now handling itemStatus property changed event, pr #14080 From what I can read, this new code is not limited to the current focus, so would report any changed item status on any element... That may be over kill. @akash07k is experiencing a much more broad problem where entire VS windows are being reported. It could also be related to itemStatus property change event, but since others cannot reproduce this that is difficult to verify. We need a detailed event log also from @akash07k

josephsl commented 2 years ago

Hi, preferably with Event Tracker add-on installed as it can track and record UIA item status event (perhaps limiting item status handling to current focus may solve parts of what’s happening. Thanks.

cary-rowen commented 2 years ago

Hi @josephsl My second log was saved after installing EventTracker and it should contain the information you need. Thanks

josephsl commented 2 years ago

Hi, based on Event Tracker log, it appears UIA item status events are accompanied by name change events. The reason NVDA says “default” is because that is item status text UIA reports (ultimately, coming from what Visual Studio (UIA server) tells UIA to report to clients such as NVDA). If item status variant issue is not resolved by switching to selective UIA event registration, then it would be better to restrict item status to focused element unless there are cases where item status should be handled from places other than focused control. At the same time, since it is really up to UIA server (Visual Studio) to communicate things about item status to NVDA and other clients, I think it would be best to keep the communication channel going between users, NV Access and contributors, and Visual Studio team inside Microsoft (of course keep in mind that Narrator can give us a good set of data points to compare event handling and such). Thanks.

akash07k commented 2 years ago

@michaelDCurran I'll generate the logs,

seanbudd commented 2 years ago

@akash07k @cary-rowen does this PR build from #14136, fix your respective issues?

@cary-rowen - I expect this PR to resolve the problem you describe @akash07k - it would be helpful to know as well, but I don't expect this build to solve the problem

cary-rowen commented 2 years ago

看上去，您提供的构建并没有解决我提到的问题。 NVDA is still announcing some redundant information, not just "default". Below is the log after installing the EventTracker add-on. nvda-log.txt

michaelDCurran commented 2 years ago

@cary-rowen Is this extra info being spoken only happening in alphas, or has this always occurred? Right now we are only concerned with a regression in alpha which could be holding back 2022.4.

cary-rowen commented 2 years ago

Hi, @michaelDCurran The issue appeared in alpha-26387 and later

codeofdusk commented 2 years ago

That's... quite odd.

Maybe we should add VS to the textChange whitelist?

cary-rowen commented 2 years ago

After making the following settings in the NVDA Advanced Settings panel: Registration for UI Automation events and property changes:

"Global" or "Automatic (prefer selective)"

NVDA's speech lag more than five seconds in WPF project in VS2022. Duplicate reporting of 'default' has been fixed in alpha-26572. But we still can't work with VS in this case due to huge speech lag.

codeofdusk commented 2 years ago

NVDA's speech lag more than five seconds in WPF project in VS2022.

@cary-rowen Is this lag also observed in NVDA 2022.3?

cary-rowen commented 2 years ago

@codeofdusk Yes, 2022.3Beta4 will also present this speech lag if the "Enable selective registration for UI Automation events and property changes" checkbox is not checked.

codeofdusk commented 2 years ago

Yes, 2022.3Beta4 will also present this speech lag if the "Enable selective registration for UI Automation events and property changes" checkbox is not checked.

This is expected behaviour (see #11002). Selective UIA registration is now enabled by default as of NVDA 2022.4+ on Windows SV2+.

Given that the original issue reported here was fixed in PR #14136 and there are no further regressions, I recommend closing this issue.

josephsl commented 2 years ago

Hi, to clarify: Windows SV2 = Windows 11 Version 22H2 (build 22621). Thanks,

cary-rowen commented 2 years ago

But another thing I've observed is that I can also reproduce this huge speech lag by checking "Automatic (prefer selective" in the advanced settings panel. Is this also expected? If so, what is the default state of NVDA for this setting on Windows 10 21H2 (x64) build 19044.1889?

codeofdusk commented 2 years ago

But another thing I've observed is that I can also reproduce this huge speech lag by checking "Automatic (prefer selective" in the advanced settings panel. Is this also expected?

Yes if running on a Windows version below SV2.

what is the default state of NVDA for this setting on Windows 10 21H2 (x64) build 19044.1889?

The "automatic" option is global on Win10 due to bugs with task manager and emoji panel.

michaelDCurran commented 2 years ago

But to clarify, NVDA 2022.2also would show this lag in VS right? I.e. this lag has not been introduced into NVDA knowingly but then only mitigated on Windows 11 22h2 with selective registration? If I understand correctly, we've known about this lag for a long time. We want to switch to UIA selective registration by default everywhere, but we simply cannot on versions less Windows 11 22h2 as it would cause missing information. If on the other hand we've introduced something after 2022.2 which makes this lag worse (where we can't do selective registration) then this is a much more serious issue. Is this your understanding @@codeofdusk and @josephsl? Do we still think changes to UIA textChange event may play into this at all?

codeofdusk commented 2 years ago

But to clarify, NVDA 2022.2also would show this lag in VS right?

Yes, I'd think so.

this lag has not been introduced into NVDA knowingly but then only mitigated on Windows 11 22h2 with selective registration?

To my knowledge.

We want to switch to UIA selective registration by default everywhere, but we simply cannot on versions less Windows 11 22h2 as it would cause missing information.

Correct.

seanbudd commented 2 years ago

@akash07k - We're closing this as we believe this is resolved, please correct us if this is not the case and we will reopen the issue