NVDA becomes sluggish with WPF TreeView UIAutomation Events

[donnie-msft]

Hi, I work on the Microsoft Visual Studio IDE, specifically the NuGet Package Manager (repo), which is our custom Master-Detail control (WPF control). Our control has performance problems with the NVDA reader. My research has me suspecting that NVDA is being bombarded with UIA Events from our TreeView control and is processing all of them and the narration becomes unstable. In my attached log file (nvda.log), you'll notice a lot of "Ignoring event" entries.

Recently closed issue with 1Password seems similar, but I understand this won't be released for testing until May 22, 2020: https://github.com/nvaccess/nvda/issues/10508 New Issue also seems like related improvement in this area: https://github.com/nvaccess/nvda/issues/11077

I'd like to discuss our issue here to understand if it's related to these other performance improvement efforts, or if there's a separate issue.

Steps to reproduce:

I'll summarize the behavior I observed. If you need more detailed Visual Studio repro steps, please let me know.

Actual behavior:

1. Selecting a package list item in NuGet's Master list causes the Detail Pane to re-load/re-bind a TreeView with a dependency tree for that specific package (average ~10 and peak around ~90 tree nodes). This tree has grandchildren Screenshot of NuGet Package Manager, with a selected Master item (1 of 4), and the resulting Tree item (approximately 100 nodes).

2. I viewed UIAutomation Events using AccEvent.exe, and saw ~230 UIA events (see below example) from simply selecting a new item in our Master list.

3. NVDA will not render any text-to-speech until AccEvent.exe displays all the UIA events in the event viewer (30 seconds to a minute).

   1. Microsoft Narrator renders text-to-speech immediately, while UIA events continue to fire in the background.
   2. The screenshot below shows that the NVDA Focus Highlighter (blue dotted line) stays on the "Updates" tab even though I've already moved my keyboard focus to the Master listbox and selected the item, Newtonsoft.Json, there:

Expected behavior:

NVDA should follow tab-focus with the keyboard without significant delay.

System configuration

NVDA installed/portable/running from source: installed

NVDA version: 2019.3.1

Windows version: Windows 10

Name and version of other software in use when reproducing the issue: Microsoft Visual Studio 2019

Other information about your system:

Other questions

Does the issue still occur after restarting your computer? Yes

Have you tried any other versions of NVDA? If so, please report their behaviors. No

If addons are disabled, is your problem still occuring? Using plain NVDA.

Did you try to run the COM registry fixing tool in NVDA menu / tools? No

[josephsl]

CC @LeonardDer, @JCSTeh

[zersiax]

Maybe this is beating a dead horse just a little, but: Why is 230 (!) events fired for that simple selection change? It sounds to me like NVDA can certainly improve in this area if Narrator is able to persevere through that flood of events, but it also sounds like WAY too many events are fired. Do we know anything about why this is happening, and if it is a problem with:

• This particular instance of the WPF control: Unlikely, given the related issues;
• This particular version with WPF;
• This particular object within this particular version of WPF: Again, this seems unlikely.

Can we get some on the WPF side to look into this as well?

[jcsteh]

Why is 230 (!) events fired for that simple selection change? It sounds to me like NVDA can certainly improve in this area if Narrator is able to persevere through that flood of events,

That's the idea behind #11077. However, that issue is still very much in the discussion/exploration phase, especially for UIA.

Even though #10508 is caused by the same underlying problem (event flooding), the workaround I implemented for that is a hack which applies specifically to 1Password.

[zersiax]

@jcsteh right, I figured as much from looking at the comments on that issue. My point was more that fixing this in NVDA, to me, feels more like fixing a symptom rather than solving the problem. I think at this point we can safely say there's something seriously wrong in the way WPF either is implemented on the framework-level, or generally used by its developers, that causes these floods to occur. It sounds to me like fixing that underlying issue would be more productive, but I also recognize that this might take a very long time, if it ever happens, and that NVDA needs to at least minimize the impact in the mean time

[jcsteh]

Oh, I did understand your point. I was just answering the questions regarding NVDA mitigation. It is totally unacceptable that the control floods like this, and unfortunately, this kind of thing is not that uncommon with Microsoft's various UIA implementations over the years.

[donnie-msft]

Could folks elaborate on the UIA implementation problem? I'd like to engage with other teams (WPF, XAML, et al) to improve this experience if there's a problem.

My current take is that NVDA is listening to every possible PropertyChanged event that WPF fires. I'm not sure the right thing is to rewire WPF not to fire these at all. Are the suggestions above that WPF shouldn't expose PropertyChanged events for all nested controls?

How would WPF know what events an application may/may not want to consume?

[LeonarddeR]

Could folks elaborate on the UIA implementation problem? I'd like to engage with other teams (WPF, XAML, et al) to improve this experience if there's a problem.

That would be very nice.

My current take is that NVDA is listening to every possible PropertyChanged event that WPF fires.

This is correct, and far from ideal. However, @zersiax also has a point, see below.

I'm not sure the right thing is to rewire WPF not to fire these at all. Are the suggestions above that WPF shouldn't expose PropertyChanged events for all nested controls?

The question is what these property change events try to communicate. Usually, property changes are only of interest to a screen reader if they apply to the focused element, either physically (system focus) or virtually (virtual focus or, as in NVDA terminology, when it is the navigator object).

How would WPF know what events an application may/may not want to consume?

It depends a bit on what event we're talking about.

Would you be able to elaborate on what property changes are fired when the sluggishness occurs? What do these changes intent to communicate?

[Adriani90]

@donnie-msft could you elaborate on the question raised by @leonardder above please? Thank you.

[donnie-msft]

@Adriani90 Sure, I included an nvda.log in the issue description. The nvda.log shows thousands of lines of events that NVDA says are ignored. For example:

DEBUG - _UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:44:30.779) - Dummy-1 (1628): HandlePropertyChangedEvent: Ignoring event UIA_itemStatus for shouldAcceptEvent=False DEBUG - _UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:44:30.822) - Dummy-1 (1628): HandlePropertyChangedEvent: Ignoring event nameChange for shouldAcceptEvent=False DEBUG - _UIAHandler.UIAHandler.IUIAutomationPropertyChangedEventHandler_HandlePropertyChangedEvent (18:44:30.852) - Dummy-1 (1628): HandlePropertyChangedEvent: Ignoring event stateChange for shouldAcceptEvent=False

[LeonarddeR]

NVDA ignores events that aren't related to the current focus. In NVDA 2020.3 due to be released soon, the option "Enable selective registration for UI Automation events and property changes" is added that limits registration to objects of interest. It really improves performance in these controls, and ideally will be the default in the near future.

[Adriani90]

@michaelDCurran Will #14888 will finally close this issue?

[Adriani90]

@donnie-msft, @zersiax are you still having this issue with NVDA 2024.2 stable or 2024.3 beta?

[Adriani90]

We didn't get any updates from the initial author upon request since many years. I am closing as abandoned. if you are still having this issue, please comment and we can reopen.

[donnie-msft]

We didn't get any updates from the initial author upon request since many years.

@Adriani90 Hello, I see that you pinged me last week about this issue. I'm not understanding how you interpret this as me being negligent for many years. I posted as many detailed logs as possible in the original issue and replied to you in 2020.

No, I have not had time to download the update you sent 6 days ago and retest this.

Also, the "abandoned" status seems a little dismissive of the work done by several folks in this thread. There was work done by others to improve the performance in this area, see the linked Pull Requests.

[Adriani90]

However, we didn‘t really get any updated test results on how the behavior is now more than 4 years later with last versions of NVDA. I am happy to reopen this if the issue still exists. Can you provide updated test results? The linked pull requests did not close this issue, so not sure they were directly meant to fix this. But if they did, then it is apparently a lucky side effect. Von meinem iPhone gesendetAm 09.07.2024 um 01:11 schrieb Donnie Goodson @.***>:

We didn't get any updates from the initial author upon request since many years.

@Adriani90 Hello, I see that you pinged me last week about this issue. I'm not understanding how you interpret this as me being negligent for many years. I posted as many detailed logs as possible in the original issue and replied to you in 2020. No, I have not had time to download the update you sent 6 days ago and retest this. Also, the "abandoned" status seems a little dismissive of the work done by several folks in this thread. There was work done by others to improve the performance in this area, see the linked Pull Requests.

—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you were mentioned.Message ID: @.***>