UIA handler/get nearest window handle: expand the GUI thread info method to cover scenarios beside Windows Defender Application Guard

[josephsl]

Hi,

This proposal stems from several recent issues and pull requests on UIA events, notably #16862, #16871, and PR #16872:

Background:

While handling UIA events, a UIA NVDA object is constructed to house porperties such as window handle and uIA elemnet, as well as pass on the object to other event handlers such as global plugins and app modules. In most cases, the resulting NVDA object will receive a window handle coming from the sender (element), and in case it isn't present, UIAHandler.handler.getNearestWindowHandle is called to retrieve the nearest native window handle. However there are cases where window handle cannot be obtained easily either because the app is isolated from others (Windows Defender Application Guard) or the element does not include it by default (Windows 11 Voice Access with just NVDA running or input switch notification).

Is your feature request related to a problem? Please describe.

Because NVDA relies on window handle (more specifically, native window handle) when constructing NVDA objects, it causes problems when the element does not offer a window handle when raising events. A notable case is Windows 11 Voice Access (#16862) where the window handle cannot be fetched when NVDA alone is running; however, when Narrator is also run, the window handle can be retrieved. Also, the input language switch notification (coming from shell/Explorer) does not even include a UIA runtime ID, a collection of integers used to uniquely identify elements, making it more difficult to obtain the native window handle for this.

As Windows and other apps are adopting UIA notification and other events raised by elements with or without window handles, it becomes important to let NVDA at least recognize this fact and log it for debugging purposes. Ideally, NVDA and its add-ons should be offered a chance to react to events from elements without window handles. In short, this proposal seeks to extend UIA event support by not limiting ourselves to elements with defined window handles.

Describe the solution you'd like

Currently NVDA employs GUI thread information routine to obtain closest active window handle. This method is used to support Windows Defender Application Guard (WDAG) because elements inside WDAG are isolated from other apps (seen as running on remote machines). However, Microsoft is deprecating WDAG. Even though WDAG is being deprecated, NVDA developers could use lessons from our WDAG support to let NVDA obtain closest window handle via GUI thread info or a similar (or even better) mechanism.

If extending GUI thread info fetcher in UIAHandler.handler.getNearestWindowHandle, I propose the following approach:

• Create a dictionary in UIA handler (main) module to record the names of processes and window classes requiring GUI thread info or other methods to obtain window handles for UIA elements when fetching widow handles. Process names (keys) will be a string while widow class names should be a list of strings 9there might e a possibility that we need to handle multiple window classes from the same process when fetching window handles without walker routines).
• The current WDAG process name and window class name constants will be kept and become the first entry in the newly defined GUI thread info dictionary, guaranteeing backward compatibility. The WDAG constants should be marked for deprecation (maybe in 2026).
• The GUI thread info dictionary (or whatever it will be called) will also record names of processes and window classes requiring a method other than UIA tree walker to fetch active window handle.
• Modify UIAHandler.handler.getNearestWindowHandle method to consult the new dictionary instead of just WDAG constants when consulting GUI thread info, and if the process name and class names match (recorded in the dictionary), return the active window handle reported by GUI thread info, otherwise proceed to UIA tree walker method. 

Describe alternatives you've considered

Either:

• Offer a facility to handle events without constructing NVDA objects (this can break compatibility; while it can increase performance a bit, it can also lead to event flooding or not being able to let NVDA components and add-ons veto events).
• Find out how Narrator works with Voice Access so NVDA too can obtain the window handle. This may not work for input language switch notification or other elements without window handles.
• Specifically for UIA notification event handler, check for nearest window handle BEFORE constructing the UIA NVDA object (this is the algorithm employed in PR #16872) to align with other event handlers such as property changes. this, together with process ID check (and construction of an app module), allows NVDA to ask the app module and other components about handling notification events with elements without apparent native window handles. Doing so also brings a small optimization - window handle can be known earlier instead of during object construction, allowing the event handler to return early if the event cannot proceed (because window handle is unknown (current approach) or not recorded in the GUI thread ifo dictionary (proposal)).
• Others that could lead to some overhaul of UIA event handling for NVDA to cover other cases.

Additional context

See #16862 for immediate context, along with input language switching exmaple.

Thanks.

[josephsl]

Hi,

Note that GUI thread info is one possible method to handle elements without apparent window handles. As long as concrete NVDA objects (such as UIA NVDA objects) derive their power from NVDAObjects.window.Window (a window object), obtaining window handle is crucial.

Thanks.

[josephsl]

Hi,

Turns out thread info may not be a suitable solution - it mostly works on focused object. Besides, this does not resolve the primary issue: Vouice Access announcements. Feel free to reopen if people find that GUI thread info would be a suitable way to move forward.

Tanks.