zalando / SwiftMonkey

A framework for doing randomised UI testing of iOS apps
MIT License
1.95k stars 174 forks source link

Failed to get matching snapshots #83

Open onmyway133 opened 5 years ago

onmyway133 commented 5 years ago

Thanks for the library. I have an issue with the latest release

Assertion Failure: MonkeyXCTest.swift:33: Failed to get matching snapshots: Timed out while evaluating UI query.

Here is my setup

func testMonkey() {
    let monkey = Monkey(frame: app.frame)
    monkey.addDefaultUIAutomationActions()
    monkey.addXCTestTapAlertAction(interval: 100, application: app)
    monkey.monkeyAround()
}
wojciechczerski commented 5 years ago

Hey there! Is the sample App working for you? Also, are you integrating the library from CocoaPods repository, or are you referencing SwiftMonkey directly from GitHub (master)?

onmyway133 commented 5 years ago

@wojciechczerski Hi, thanks for quick response. I use CocoaPods, and I see it origins from this call addXCTestTapAlertAction

wojciechczerski commented 5 years ago

Could you try referencing SwiftMonkey in your Podfile like that:

pod 'SwiftMonkey', :git => 'https://github.com/zalando/SwiftMonkey.git'

I would like to check whether perhaps the existing code on master could solve your issue.

onmyway133 commented 5 years ago

@wojciechczerski ja will try, and also I have this issue which happens only during monkey tests

Failed to determine hittability of Button: Unable to fetch parameterized attribute XC_kAXXCParameterizedAttributeConvertHostedViewPositionFromContext, remote interface does not have this capability.

wojciechczerski commented 5 years ago

When you fetch the code from GitHub, could you also try replacing monkey.addDefaultUIAutomationActions with monkey.addDefaultXCTestPublicActions(app: application)? I know it's not really solving the issue but maybe using XCTest public action could at least get you going while the issue is being solved.

onmyway133 commented 5 years ago

@wojciechczerski seems to work, thanks

adario commented 4 years ago

@wojciechczerski any news about this issue? I'm using the GitHub version in my Podfile (currently at 2.1.1), and the matching snapshots failure is still present. Regarding your suggestion to use monkey.addDefaultXCTestPublicActions(app: application) -- that method does not appear to be available. Thank you.

wojciechczerski commented 4 years ago

@adario Hey there! Is this how your Podfile looks like?

pod 'SwiftMonkey', :git => 'https://github.com/zalando/SwiftMonkey.git'

If the monkey.addDefaultXCTestPublicActions method is not present, perhaps there is some caching issue that's preventing pulling the latest version. In such case you could try deleting the Pods directory and running pod install again.

adario commented 4 years ago

@wojciechczerski Hey there, thanks for the quick reply — yes, my Podfile contains the :git statement to locate the pod, so I'm on the latest (I assume: pod install reports version 2.1.1). The method is indeed missing: I've looked at the sources, and couldn't find it.

wojciechczerski commented 4 years ago

Have you tried deleting the Pods directory so that SwiftMonkey gets reinstalled. Another thing to try could be to delete Podfile.lock. Alternatively, just to be sure you are getting the lastest sources, you could try adding this to your Podfile (as a test):

pod 'SwiftMonkey', :git => 'https://github.com/zalando/SwiftMonkey.git', :commit => '68f49bb69381103224b0e77ab615baac8b1fd750'

Please let me know whether that helps.

adario commented 4 years ago

@wojciechczerski adding the commit ID in the Podfile (after manually removing Podfile.lock and Pods) seems to work — that is, the addDefaultXCTestPublicActions method is now visible... Is the method supposed to work on actual devices, or only on the simulator? Thank you.

adario commented 4 years ago

It looks like the public actions do work on actual devices — cool! Is it possible to add extra actions besides tap, press and drag? Thank you.

wojciechczerski commented 4 years ago

Glad to hear it worked for you! As for adding extra actions - I am not sure. SwiftMonkey used to support quite a lot of actions, this was however possible due to usage of private APIs. These APIs do not exist anymore therefore public XCTest actions are used. Maybe it would be possible to combine these actions to simulate a pinch or rotate gestures, but I haven't checked that yet (it's still on my TODO).

adario commented 4 years ago

@wojciechczerski I've tried adding public XCTest actions for swipe and rotate: unfortunately, they're not very useful. That is, they are executed correctly, as well as shown by MonkeyPaws, but since they're run via the application XCUIElement instead of XCUICoordinate, they don't produce the expected results because of truly counterintuitive locations... For instance, the rotate gesture is applied to the edges of the screen, instead of around the center of the screen, which obviously results in no rotation at all. Conversely, the swipe gestures are applied around the screen center, instead of from the screen edges.