Support links inside UITextView attributed strings

[haitaoli]

If UITextView content contains links, EG is not able to detect them. The whole string is still accessible as static text. Accessibility Explorer list the links as child elements, and XCUITest framework can detect them as well, even though UITextView doesn't seem to implement any of the methods in UIAccessibilityContainer protocol. Here is what Accessibility Explorer shows:

[tirodkar]

Can you give me a sample Text View string with a link? Just so we know we're working with the same test?

[haitaoli]

The links need to be set up in code. I'll try to build some sample code for this.

[tirodkar]

That would be great! Thanks a lot @haitaoli

[haitaoli]

@tirodkar sorry for taking so long. Here is the project: https://github.com/haitaoli/EarlGreyAttributedTextTests

[tirodkar]

Thanks @haitaoli! We'll test it out and get back with updates.

[VirtualFox0]

I have the same issue without Earl Grey. Links within UITextViews have a strange behaviour. See the following link: https://stackoverflow.com/questions/41353959/xctest-how-to-tap-on-url-link-inside-uitextview/46530038#46530038

[tirodkar]

@ISCHI, we spoke to apple devs about this at WWDC, but it didn't look like they were planning to support this with the Accessibility Inspector. I'll file a radar and paste the link here for more details.

[gearedupcoding]

@tirodkar Is there any update on this? Thank you!

[tirodkar]

No details back on this. Any way to get this with a custom matcher?

[fumoboy007]

As of iOS 13.4, this matcher works:

grey_allOf(grey_accessibilityLabel(linkText),
           grey_accessibilityTrait(UIAccessibilityTraitLink),
           nil)

To activate the link, we also need the following:

/**
 * Initializes an action that performs the default accessibility action.
 *
 * @remarks An example situation where this is needed is multi-line text links. As of iOS 14.4, the
 * accessibility element representing the text link does not have a text-aware activation point; the
 * activation point is simply the center of the accessibility frame. However, the center may not be
 * tappable if the text link starts near the end of the first line and ends near the beginning of
 * the second line.
 *
 * @return An instance of GREYAction.
 */
+ (id<GREYAction>)actionForPerformDefaultAccessibilityAction {
  return [GREYActionBlock
      actionWithName:@"Default accessibility action"
        performBlock:^(NSObject *element, NSError *__strong *errorOrNil) {
          __block BOOL success = NO;
          grey_dispatch_sync_on_main_thread(^{
            success = [element accessibilityActivate];
          });

          if (!success) {
            I_GREYPopulateError(
                errorOrNil, kGREYInteractionErrorDomain, kGREYInteractionActionFailedErrorCode,
                @"The element did not successfully activate the default accessibility action.");
          }
          return success;
        }];
}

[tirodkar]

This is neat. Do we need to activate the link within the container or do we need to perform a tap on it as well? If it's the former then we could add something for this.

[fumoboy007]

What do you mean by “activate the link within the container”?

---

As of iOS 13.4, this is what the accessibility hierarchy looks like (excluding irrelevant elements):

<UITextView:0x7fb7d6892e00; isAccessible=N; AX.value='Regular text. Link 1. Link 2.'; AX.frame={{24, 221}, {327, 17}}; AX.activationPoint={29, 221}; AX.traits='UIAccessibilityTraitStaticText'; AX.focused='N'; frame={{24, 149}, {327, 17}}; opaque; alpha=1; text='Regular text. Link 1. Link 2.'>
|--<_AXUITextViewParagraphElement:0x600003244d20; isAccessible=Y; AX.hint='Double tap to activate embedded link'; AX.value='Regular text. Link 1. Link 2.'; AX.frame={{24, 221}, {171.923828125, 16.70703125}}; AX.activationPoint={109.9619140625, 229.353515625}; AX.traits='UIAccessibilityTraitLink,UIAccessibilityTraitStaticText'; AX.focused='N'>
|  |--<UIAccessibilityLinkSubelement:0x6000035271b0; isAccessible=Y; AX.label='Link 1.'; AX.frame={{109.298828125, 221}, {40.373046875, 16.70703125}}; AX.activationPoint={129.4853515625, 229.353515625}; AX.traits='UIAccessibilityTraitLink'; AX.focused='N'>
|  |--<UIAccessibilityLinkSubelement:0x600003527200; isAccessible=Y; AX.label='Link 2.'; AX.frame={{153.458984375, 221}, {42.46484375, 16.70703125}}; AX.activationPoint={174.69140625, 229.353515625}; AX.traits='UIAccessibilityTraitLink'; AX.focused='N'>

I was wrong about UIAccessibilityLinkSubelement not having a text-aware accessibility frame/activation point. It actually has logic to use the rect of the first line as the accessibility frame. It uses that logic only if the accessibility container is a UITextView; however, the actual container is _AXUITextViewParagraphElement. I think this is an iOS bug. Once fixed, we can use grey_tap() as expected. In the meantime, we need to use my proposed grey_performDefaultAccessibilityAction().

[fumoboy007]

iOS bug reported in FB8997424 (tracked at Google in b/179745324). Note that this bug affects Xcode UI Tests as well.

[fumoboy007]

Hmm the above matcher only works for non-editable text views. Editable text views do not have UIAccessibilityLinkSubelement elements in its accessibility hierarchy.