Closed josephsl closed 1 month ago
Hi,
Note that GUI thread info is one possible method to handle elements without apparent window handles. As long as concrete NVDA objects (such as UIA NVDA objects) derive their power from NVDAObjects.window.Window (a window object), obtaining window handle is crucial.
Thanks.
Hi,
Turns out thread info may not be a suitable solution - it mostly works on focused object. Besides, this does not resolve the primary issue: Vouice Access announcements. Feel free to reopen if people find that GUI thread info would be a suitable way to move forward.
Tanks.
Hi,
This proposal stems from several recent issues and pull requests on UIA events, notably #16862, #16871, and PR #16872:
Background:
While handling UIA events, a UIA NVDA object is constructed to house porperties such as window handle and uIA elemnet, as well as pass on the object to other event handlers such as global plugins and app modules. In most cases, the resulting NVDA object will receive a window handle coming from the sender (element), and in case it isn't present, UIAHandler.handler.getNearestWindowHandle is called to retrieve the nearest native window handle. However there are cases where window handle cannot be obtained easily either because the app is isolated from others (Windows Defender Application Guard) or the element does not include it by default (Windows 11 Voice Access with just NVDA running or input switch notification).
Is your feature request related to a problem? Please describe.
Because NVDA relies on window handle (more specifically, native window handle) when constructing NVDA objects, it causes problems when the element does not offer a window handle when raising events. A notable case is Windows 11 Voice Access (#16862) where the window handle cannot be fetched when NVDA alone is running; however, when Narrator is also run, the window handle can be retrieved. Also, the input language switch notification (coming from shell/Explorer) does not even include a UIA runtime ID, a collection of integers used to uniquely identify elements, making it more difficult to obtain the native window handle for this.
As Windows and other apps are adopting UIA notification and other events raised by elements with or without window handles, it becomes important to let NVDA at least recognize this fact and log it for debugging purposes. Ideally, NVDA and its add-ons should be offered a chance to react to events from elements without window handles. In short, this proposal seeks to extend UIA event support by not limiting ourselves to elements with defined window handles.
Describe the solution you'd like
Currently NVDA employs GUI thread information routine to obtain closest active window handle. This method is used to support Windows Defender Application Guard (WDAG) because elements inside WDAG are isolated from other apps (seen as running on remote machines). However, Microsoft is deprecating WDAG. Even though WDAG is being deprecated, NVDA developers could use lessons from our WDAG support to let NVDA obtain closest window handle via GUI thread info or a similar (or even better) mechanism.
If extending GUI thread info fetcher in UIAHandler.handler.getNearestWindowHandle, I propose the following approach:
Describe alternatives you've considered
Either:
Additional context
See #16862 for immediate context, along with input language switching exmaple.
Thanks.