Windows 11 UI: recognize DesktopWindowXamlSource objects as UIA elements

[josephsl]

Hi,

Stemming from nvaccess/nvda#13506:

Windows 11 uses a newer GUI framework (WinUI, to be exact) for UI elements, including Taskbar, systray, and parts of varous apps such as Windows 11 Notepad. These elements have the name of "DesktopWindowXamlSource", indicating inclusino of XAML controls inside classic (Win32) apps. At the moment items inside these windows such as Taskbar items are not detected properly when using the mouse or touch interaction, traced to the fact that NVDA sees the XAML windows as IAccessible (MSAA) elements. Reclassifying these windows as UIA resolves this, allowing NVDA to detect UI elements.

There are two wasy to resolve this from the add-on:

1. At the global plugin level, add the new window class names to NVDA Core's good UIA window class names list (a tuple, actually). This involves converting the NVDA Core good UIA classes to a list, add the new class names, and convert it back to a tuple. This will allow NVDA to work with these controls from everywhere.
2. At the app module level, define isGoodUIAWindow method that will return "true" if the new class names are seen through use of user32.dll::GetClassNameW function (File Explorer app module from NVDA Core is a good example).

For now use a combination of the two:

1. File Explorer/Taskbar: global plugin method wil be used becasue there are add-ons that add things on top of File Explorer app module.
2. Notepad/menu bar: use app module method.

Ideally a solution that works across aps is better, but for the purposes of demonstrating a possible solution, this will do.

Thanks.

[josephsl]

Hi,

Should have been marked as resolved in add-on 22.06 but forgot to do so - app module solution implemented in NVDA 2022.2. Thanks.