SwiftUI Views with ScrollViews for iOS 14 don't record Voice Over Descriptions

[fbernutz]

Something seems to have changed from iOS 13 to iOS 14 with SwiftUI ScrollViews.

When snapshotting a SwiftUI View with a ScrollView (only iOS 14), no Accessibility Nodes can be found. I tried to debug this and found differences in the objects when the SwiftUI View had a ScrollView and when it had no ScrollView.

iOS 13: Elements inside a ScrollView are of type NSObject

Screenshot

iOS 14: Elements inside a ScrollView are of type UIView (SwiftUI_UIShapeHitTestingView)

Screenshot

This results in no accessibility nodes for the iOS 14 snapshot.

iOS 13	iOS 14

I've tried to fix this but honestly I have no idea how 🙈

I've also added a draft PR #34 with two new snapshot tests where you can see the resulting snapshot for these scenarios. If there are any more information needed to look into this, I'm happy to help.

[NickEntin]

Thanks for the report @fbernutz! I'll run through it in the debugger and see if I can figure out what's going on.

[NickEntin]

Hmm, this is an interesting edge case. It seems like SwiftUI isn't quite implementing the UIAccessibility API the way I understand it's meant to work. The UIHostingView now acts as an accessibility container with the scroll view as its only element. Because the scroll view is not an element (as expected), this results in there being no elements shown when we parse through the hierarchy.

(lldb) po self
<_TtGC7SwiftUI14_UIHostingViewV25AccessibilitySnapshotDemo25SwiftUIViewWithScrollView_: 0x...; frame = (0 0; 375 812); gestureRecognizers = <NSArray: 0x...>; layer = <CALayer: 0x...>>

(lldb) po accessibilityElements
▿ 1 element
  ▿ 0 : <SwiftUI.HostingScrollView: 0x...; baseClass = UIScrollView; frame = (172.667 0; 30 812); anchorPoint = (0, 0); clipsToBounds = YES; gestureRecognizers = <NSArray: 0x...>; layer = <CALayer: 0x...>; contentOffset: {0, -44}; contentSize: {30, 488}; adjustedContentInset: {44, 0, 34, 0}>

(lldb) po accessibilityElements[0].isAccessibilityElement
false

VoiceOver seems to recover from this though, and keeps searching for other elements. I need to keep digging to figure out how it determines whether to accept that an accessibility container is empty vs. continuing to search for other elements.

[fbernutz]

Maybe this behavior is a bug in iOS 14.0? I haven't tested it with one of the iOS 14 betas. Will do later.

[fbernutz]

the issue exists in iOS 14.1 (Xcode 12.1 GM) as well... BUT in iOS 14.2 beta (Xcode 12.2 beta 3) it seems to be fixed, the snapshot looks great there.

[NickEntin]

Did some testing on iOS 13 and there's definitely some existing behavior around accessibility containers that we aren't getting right. It looks like nesting accessibility containers will include all of the elements in the hierarchy, but I still need to do some experimenting to figure out how it deals with ordering the elements and some of the edge cases.

[NickEntin]

Working on debugging the nested container handling in #46

[NickEntin]

So, from what I can tell, we're actually handling nested accessibility containers correctly, with the exception of some edge cases around ordering that shouldn't affect this case.

I ran the scroll view test you added in #34 on iOS 14.1 and stepped through the parser, and got this:

UIView, subviews (1)

-> SwiftUI.UIHostingView<SwiftUIViewWithScrollView>, accessibility container (1)

   -> SwiftUI.HostingScrollView, subviews (3)

      -> SwiftUI.HostingScrollView.PlatformGroupContainer, subviews (8)

         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)
         -> SwiftUI.UIShapeHitTestingView, subviews (0)

      -> UIScrollViewScrollIndicator, hidden

      -> UIScrollViewScrollIndicator, hidden

The hosting view acts as an accessibility container, vending only the scroll view. The scroll view isn't an element itself or an accessibility container, so we look at its subviews. The first subview is the PlatformGroupContainer, in which we look at its subviews again, but none are elements. The other two subviews (the scroll indicators) are hidden, so we can ignore them. By the end, we've parsed the entire hierarchy without finding any accessibility elements.

Testing this on 14.2, the PlatformGroupContainer serves as an accessibility container, vending a series of SwiftUI.AccessibilityNode instances:

UIView, subviews (1)

-> SwiftUI.UIHostingView<SwiftUIViewWithScrollView>, accessibility container (1)

   -> SwiftUI.HostingScrollView, subviews (3)

      -> SwiftUI.HostingScrollView.PlatformGroupContainer, accessibility container (8)

         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)
         -> SwiftUI.AccessibilityNode (element)

      -> UIScrollViewScrollIndicator, hidden

      -> UIScrollViewScrollIndicator, hidden

It definitely seems like this was a SwiftUI bug in iOS 14.{0,1} that's now fixed in iOS 14.2. I'm not sure how exactly VoiceOver was still able to find the elements (presumably it must communicate with SwiftUI in a different way), but I don't see anything through the UIAccessibility API that would give us access to them. 😕

[fbernutz]

Thanks for your analysis. As iOS 14.2 is already officially released, I'll change my snapshot references to 14.2 soon, so there's no need for a workaround for the two other iOS versions I think.

[NickEntin]

I just added snapshots for iOS 14.2 (see #52) if you want to add some regression tests for this. Happy to close out this issue now, or anything else you think needs to be addressed?

[fbernutz]

Awesome, just saw that! Thanks for all the support and feel free to close this issue.