kif-framework / KIF

Keep It Functional - An iOS Functional Testing Framework
Other
6.2k stars 909 forks source link

Xcode 13.4: NSFunctionExpression with selector is forbidden. #1260

Closed erikkerber closed 2 years ago

erikkerber commented 2 years ago

When running with Xcode 13.4, running with KIF can crash when interacting with an a11y element:

#6  0x00000001125529bc in __110+[UIAccessibilityElement(KIFAdditions) accessibilityElement:view:withElementMatchingPredicate:tappable:error:]_block_invoke at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Additions/UIAccessibilityElement-KIFAdditions.m:87
#7  0x000000011255cb0c in -[UIView(KIFAdditions) accessibilityElementMatchingBlock:notHidden:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Additions/UIView-KIFAdditions.m:147
#8  0x000000011255ca40 in -[UIView(KIFAdditions) accessibilityElementMatchingBlock:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Additions/UIView-KIFAdditions.m:135
#9  0x0000000112554ec8 in -[UIApplication(KIFAdditions) accessibilityElementMatchingBlock:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Additions/UIApplication-KIFAdditions.m:67
#10 0x00000001125527f8 in +[UIAccessibilityElement(KIFAdditions) accessibilityElement:view:withElementMatchingPredicate:tappable:error:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Additions/UIAccessibilityElement-KIFAdditions.m:86
#11 0x000000011253d380 in __89-[KIFUITestActor waitForAccessibilityElement:view:withElementMatchingPredicate:tappable:]_block_invoke at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFUITestActor.m:144
#12 0x0000000112535bc4 in -[KIFTestActor tryRunningBlock:complete:timeout:error:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFTestActor.m:88
#13 0x0000000112535e80 in -[KIFTestActor runBlock:complete:timeout:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFTestActor.m:111
#14 0x0000000112535f80 in -[KIFTestActor runBlock:complete:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFTestActor.m:118
#15 0x0000000112536064 in -[KIFTestActor runBlock:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFTestActor.m:128
#16 0x000000011253d30c in -[KIFUITestActor waitForAccessibilityElement:view:withElementMatchingPredicate:tappable:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFUITestActor.m:143
#17 0x000000011253cffc in -[KIFUITestActor waitForAccessibilityElement:view:withIdentifier:tappable:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/Classes/KIFUITestActor.m:131
#18 0x000000011253aea8 in -[KIFUITestActor(IdentifierTests) waitForViewWithAccessibilityIdentifier:tappable:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/IdentifierTests/KIFUITestActor-IdentifierTests.m:48
#19 0x000000011253abe8 in -[KIFUITestActor(IdentifierTests) waitForViewWithAccessibilityIdentifier:] at /var/tmp/_bazel_ekerber/dd323d2a4ff53820c11ccc0d5aebe791/external/com_github_kif-framework_kif/Sources/KIF/IdentifierTests/KIFUITestActor-IdentifierTests.m:20

Similar issue in Detox: https://github.com/wix/Detox/issues/3384

asafkorem commented 2 years ago

Hey @erikkerber, this is related to the iOS version, not Xcode. For the meanwhile, I suggest running your tests on Simulator with iOS version <= 15.0.

erikkerber commented 2 years ago

I assume it began with the iOS 15.5 simulator then, which is part of (and default for) Xcode 13.4. I then think most will run into this issue when they update to Xcode 13.4.

I'll test when I get a chance, but if your'e right I'm guessing that developers can still use iOS 15, it just has to be <15.5.

asafkorem commented 2 years ago

You can see how I fixed a similar error in Detox as a reference: https://github.com/wix/Detox/pull/3420.

babbage commented 2 years ago

I am experiencing the same issue. My KIF tests run fine in Xcode 13.3.1 with iOS 15.4 simulators, but crash with a NSPredicate: NSFunctionExpression with selector 'accessibilityIdentifier' is forbidden. error on

+ (BOOL)accessibilityElement:(out UIAccessibilityElement **)foundElement view:(out UIView **)foundView withElementMatchingPredicate:(NSPredicate *)predicate tappable:(BOOL)mustBeTappable error:(out NSError **)error; in UIAccessibilityElement-KIFAdditions.

babbage commented 2 years ago

Have confirmed that if you download the iOS 15.4 Simulator in Xcode 13.4, tests then pass that otherwise crash with the error above. So a partial work-around for people who want to use the latest Xcode in the meantime is to test on the iOS 15.4 Simulator.

I'd be happy to make a pull request for this if I knew how to fix it but am a bit unclear at this stage. Clearly in iOS 15.5 (or at least the Simulator for it) the format of the NSPredicate being evaluated is being treated as invalid. I've not previously used an NSPredicate as a Block Predicate the way that it is utilised in this code, and was quite interested to see how that worked... The only other repo on Github that has any issues with the error as described here is the wix/Detox one already linked in this thread, I couldn't find other examples. The particular way the NSPredicate is changed there presumably gives a guide but I'm a little unclear what we're solving for here at present. Is it that a Block Predicate needs to be abandoned? Or can it be updated to be compatible? I don't want to re-write significant parts of KIF only to find there is a much more localised fix.

No indication I've been able to find at this stage as to whether this is some intended more strict test or just a frank bug in this iOS version. Obviously we'll be expecting new versions of Xcode and iOS in beta next week, so will be interesting to see if this bug reproduced there also.

asafkorem commented 2 years ago

@erikkerber, @babbage I was able to reproduce and find it in KIF very quickly. Here's the fix: https://github.com/kif-framework/KIF/pull/1261, you're welcome 😄

KentLottis-jwn commented 2 years ago

Thank you @asafkorem - I concur with your diagnosis and your fix. I still don't understand why this cropped up: has this predicate form always been illegal and only now is Apple enforcing it? Or did they break something?

Regardless, it seems like the block-based implementation is more efficient anyway, since it skips the text-parsing step.

AhmedEid commented 2 years ago

@asafkorem Thank you for fixing this! Can you please release a version and add a tag to the commit that contains this fix? https://github.com/kif-framework/KIF/tags

dostrander commented 2 years ago

@AhmedEid I'm updating now, just going through the whole cocoapods run around with it :)

dostrander commented 2 years ago

Tagged and released to cocoapods

victorgallegomodmed commented 1 year ago

This is having some unwanted side effects in the report, that looks like this:

SomeTests.swift:215: error: -[MyAutomationTests.SomeTests testMethod] : The step timed out after 10.00 seconds: Could not find element with BLOCKPREDICATE(0x600002490180) (KIFFailureException)

Here is the set of methods that this traces to:

- (void)waitForAccessibilityElement:(UIAccessibilityElement * __autoreleasing *)element view:(out UIView * __autoreleasing *)view withElementMatchingPredicate:(NSPredicate *)predicate tappable:(BOOL)mustBeTappable

{

    [self runBlock:^KIFTestStepResult(NSError **error) {

        return [UIAccessibilityElement accessibilityElement:element view:view withElementMatchingPredicate:predicate tappable:mustBeTappable error:error] ? KIFTestStepResultSuccess : KIFTestStepResultWait;

    }];

}

and usage of error is like this:

if (!element) {
        if (error) {
            *error = [self errorForFailingPredicate:predicate];
        }
        return NO;
    }

Any modification we can make to get the block predicate to print correctly? Not sure it matters, but using Xcode 14.0, Macbook M1, but also present on Intel based systems.