search

cashapp / AccessibilitySnapshot

Easy regression testing for iOS accessibility
Apache License 2.0
541 stars 68 forks source link

Snapshots do not reflect the use of accessibilityUserInputLabels #59

Closed annahayhurst closed 11 months ago

annahayhurst commented 3 years ago

As of iOS 13, the accessibilityUserInputLabels array can be used as an alternative to a singular accessibilityLabel on an accessibility element. As per the dev docs, iOS respects this array as first priority and falls back to the standard label if it returns as empty or nil.

In our app, we have a button that sets this property with an array of two strings if the property is available. When AX snapshots are taken, they do not load the label(s) from the array. I would expect the snapshot to either reflect the first element (in our case, "Fast Forward") or to show some representation of the array, but it instead shows the underlying button name forward15 as if we never set any AX info.

The app itself handles the array fine (we use it for voice input) and if I set the usual accessibilityLabel the snapshot shows that label. Tests are running on iOS 14.3.

NickEntin commented 3 years ago

Great catch, Anna! Thanks for opening this issue. We should definitely add support for accessibilityUserInputLabels. I think the right behavior here is to mirror the way VoiceOver would describe the element based on the given labels.

Are you interested in opening a PR to update this behavior?

fruitcoder commented 3 years ago

Isn't accessibilityUserInputLabels currently only used for VoiceControl? We use it as well and VoiceOver still just reads the a11yLabel on views/accessibility elements that have both properties set. I think it would make sense to be able to choose which labels to use and to provide a similar display mode like "accessibilityLabelDisplayMode" where you can choose whenOverriden/a11Label/a11yInputLabels. whenOverriden would use the input labels when accessibilityUserInputLabels is set and contains more than one item or just one item that is different from the a11y label.

NickEntin commented 3 years ago

Looks like you're right, Alex. I did some tests on iOS 14.3 and VoiceOver doesn't seem to change any behavior based on the accessibilityUserInputLabels, regardless of whether the accessibilityLabel is set or not.

Rather than replacing the accessibilityLabel in the snapshot, I think we should instead show any user input labels in addition to the VoiceOver description. I'll mock up some designs for this, but I think the key will be making it clear that these are individual items that can be used to refer to the element, rather than used to make a composite description.