MobileNativeFoundation / bluepill

Bluepill is a reliable iOS testing tool that runs UI tests using multiple simulators on a single machine
BSD 2-Clause "Simplified" License
3.18k stars 231 forks source link

Bluepill fails #480

Open AlexanderPan opened 3 years ago

AlexanderPan commented 3 years ago

bluepill version: 5.8.0 xcode version: 12.4

run bluepill -c config.json

{
   "xctestrun-path": "./output/Build/Products/xxx_iphonesimulator14.4-x86_64.xctestrun",
   "output-dir": "./output/bluepill/",
   "runtime": "iOS 14.4",
   "num-sims": 6
}
2021-03-04 11:58:18.372 bluepill[8104:2796488] Debug Enabled == 0
{8104} 20210304.115819 [  INFO  ] (BLUEPILL) Using xctestrun configuration
{8104} 20210304.115819 [  ERROR ] (BLUEPILL) Could not extract bundleID: (null)
2021-03-04 11:58:19.805 bluepill[8104:2796488] *** Terminating app due to uncaught exception 'NSInvalidArgumentException', reason: '*** -[__NSDictionaryM setObject:forKeyedSubscript:]: key cannot be nil'
*** First throw call stack:
(
    0   CoreFoundation                      0x00007fff204876af __exceptionPreprocess + 242
    1   libobjc.A.dylib                     0x00007fff201bf3c9 objc_exception_throw + 48
    2   CoreFoundation                      0x00007fff2053ba9a -[__NSCFString characterAtIndex:].cold.1 + 0
    3   CoreFoundation                      0x00007fff20547a1f -[__NSDictionaryM setObject:forKeyedSubscript:].cold.2 + 0
    4   CoreFoundation                      0x00007fff203d587e -[__NSDictionaryM setObject:forKeyedSubscript:] + 999
    5   bluepill                            0x00000001048e5439 +[BPXCTestFile BPXCTestFileFromDictionary:withTestRoot:andXcodePath:andError:] + 1777
    6   bluepill                            0x00000001048d1056 +[BPApp testsFromXCTestRunDict:andXCTestRunPath:andXcodePath:withError:] + 574
    7   bluepill                            0x00000001048d160e +[BPApp appWithConfig:withError:] + 1049
    8   bluepill                            0x00000001048c8689 main + 971
    9   libdyld.dylib                       0x00007fff20330631 start + 1
)
libc++abi.dylib: terminating with uncaught exception of type NSException
fish: 'bluepill -c config.json' terminated by signal SIGABRT (Abort)
ravimandala commented 3 years ago

@AlexanderPan We use Bazel variant which uses test-plan-path instead of xctestrun-path and v5.8.0 works fine. Let me try the Xcode version using xctestrun-path and get back to you. Meanwhile, can you examine the contents of file referred by bp/src/SimulatorHelper.m#L157 and why the NSDictionary is coming out to be (null) for you?

chenxiao0228 commented 3 years ago

@AlexanderPan would you be able to share a redacted version of this file:

"./output/Build/Products/xxx_iphonesimulator14.4-x86_64.xctestrun" ?

AlexanderPan commented 3 years ago

@chenxiao0228

xcodebuild build-for-testing \
    -enableCodeCoverage YES \
    -workspace xxx.xcworkspace \
    -scheme xxxUITests \
    -destination "platform=iOS Simulator,name=iPhone 12 Pro Max" \
    -configuration Tests \
    -derivedDataPath 'output/' \
    | xcpretty

iphonesimulator14.4-x86_64.xctestrun.zip

AlexanderPan commented 3 years ago

@ravimandala

https://github.com/linkedin/bluepill/blob/master/bp/src/BPXCTestFile.m#L171

My .xctestrun file DependentProductPaths contains apple watch extension. At TESTROOT/Tests-iphonesimulator/AppleWatchBarImg not found Info.plist.

When I remove DependentProductPaths apple watch extension path. It work.

Wrong path : TESTROOT/Tests-iphonesimulator/AppleWatchBarImg Correct path: TESTROOT/Tests-watchsimulator/AppleWatchBarImg.app

parikhparth23 commented 1 year ago

@ravimandala i am building my app using the following command: bazelisk build --config=Debug /MyApp

What is the value I need to use for test-plan-path

I am able to generate .xctestrun for xctestrun-path but I am not sure what the value is for the key test-plan-path