UI Automation hit testing on CollectionView items hangs Maui app

[gbarkerz]

Description

Repro app is at https://github.com/gbarkerz/MyVeryFirstMauiApp.

UI Automation hit testing on the CollectionView items hangs the app. So use of hit testing with Accessibility Insights for Windows or use of the Narrator screen reader with touch is broken.

Steps to Reproduce

1. Build and run the repro app on Windows.
2. Run Accessibility Insights for Windows with its default setting of "Selection by mouse hover" on.
3. Move the mouse over the CollectionView items in the repro app.

Result: Before long the repro app will become unresponsive.

My version of dotnet is 6.0.400-preview.22330.6.

Version with bug

Unknown/Other (please specify)

Last version that worked well

Unknown/Other

Affected platforms

Windows

Affected platform versions

net6.0-windows10.0.19041.0

Did you find any workaround?

No response

Relevant log output

No response

[sohelmulani1998]

GitHubTags:

.NET MAUI; #A11ySev1; #A11yMAS; #A11yTCS; #Benchmark; #DesktopApp; #Win11; #WCAG2.1.1; #GH_.NETMAUI_Win32_Apr2022; #AILimited​; #Win32; #Keyboard;

[PureWeen]

@gbarkerz It looks like this issue is caused by the Frame class. So, inside FrameRenderer we are returning a FrameworkAutomationPeer.

https://github.com/dotnet/maui/blob/main/src/Controls/src/Core/Compatibility/Handlers/Windows/FrameRenderer.cs#L26-L35

The purpose of this appears to be so the elements are discoverable by our selenium tests.

I noticed that WinUI also does the same thing.

Do you think it's odd that this code is causing Insights to crash? or does that seem like something that might happen and we should just remove/rework the code from FrameRenderer ?

[gbarkerz]

@PureWeen Removing an AutomationPeer which has a negative impact on the end-user experience, when the AutomationPeer is only used for testing purposes, seems exactly the right thing to do. Thanks very much for doing that, I'm really looking forward to having access to this fix.

As to whether a Frame should ever return an AutomationPeer perhaps depends on whether that's the best way to enable the desired end-used experience. At https://learn.microsoft.com/en-us/windows/apps/design/accessibility/custom-automation-peers#built-in-automation-peer-classes, it lists some UWP classes that don't implement AutomationPeers because they provide a "layout behavior that is visual only". Following that logic, it would seem appropriate for the MAUI Frame to also not implement an AutomationPeer. But sometimes the app builder may know that (say) a Grid is used to provide both visual layout and semantic grouping, and as such, the Grid needs to be able expose a UIA Name property appropriate to the grouping. If a Frame would ever be used to provide semantic grouping, then it must also be able to expose a custom UIA Name property set by the app builder. But perhaps in practice this will never really be required, because a Frame could be used in conjunction with another container element which can expose the required UIA Name. And besides, based on https://learn.microsoft.com/en-us/dotnet/maui/user-interface/controls/frame, it seems that use of Frames in a MAUI app is discouraged anyway.

So all in all, the fix you've made here seems great! 🙂