search

Flank / flank

:speedboat: Massively parallel Android and iOS test runner for Firebase Test Lab
https://firebase.community/
Apache License 2.0
672 stars 113 forks source link

Using test plans for iOS test execution on Firebase #2346

Open braindonor opened 1 year ago

braindonor commented 1 year ago

I'm struggling to configure Flank to execute just the tests in my selected test plan. Local test execution is fine, but when I execute my tests on Firebase using Flank, all the tests in my project are executed. The "OnlyTestIdentifiers" section in my xctestrun file contains the tests selected in my test plan, but this appears to be ignored in my test execution.

Any pointers to how I can debug the cause would be appreciated. I have made some attempts to debug, but I'm not an expert with Kotlin, and don't fully understand the flow through the code. It seems that NSDictionary.setOnlyTestIdentifiers overwriting the full list of tests. I guess there could be some issue on the Firebase side too.

I'm using the latest Flank, and Xcode 14.2.

braindonor commented 1 year ago

I think I've worked out what's going on now. The plan filter is working for SkipTestIdentifiers, but only when the class and method are specified in the xctestrun skip list. Some of my entries have class only.

The OnlyTestIdentifiers option isn't working at all for me.