Switch to iOS Test Runner for all bazel-based test

dennycd commented 2 years ago

From a number of run samples, it looks like our default iOS host setup has large level of launch failures. Investigate switching to ios test runner based setup to check if this resolves the issue.

Sample host launcher failure

cc @HannahShiSFB

jtattermusch commented 2 years ago

Adding log here for visibility:

==================== Test output for //src/objective-c/tests:UnitTests:
2022-05-17 15:56:05,778 Will consider the test as test type xctest to run.
2022-05-17 15:56:05,778 Generating xctestrun file.
2022-05-17 15:56:07,481 Creating a new simulator:
Name: New-iPhone 11 Pro Max-13.3
OS: iOS 13.3
Type: iPhone 11 Pro Max
2022-05-17 15:56:07,648 Created new simulator 9C75AAC8-88CE-4003-9858-1C4A4D3C4BD4.
2022-05-17 15:56:07,648 Running test-without-building with device 9C75AAC8-88CE-4003-9858-1C4A4D3C4BD4
User defaults from command line:
    IDEBuildOperationResultBundlePath = /Volumes/BuildData/tmpfs/tmp/bazel/sandbox/darwin-sandbox/6160/execroot/com_github_grpc_grpc/bazel-out/darwin-fastbuild/testlogs/src/objective-c/tests/UnitTests/test.outputs/test.xcresult
    IDEDerivedDataPathOverride = /Volumes/BuildData/tmpfs/tmp/bazel/sandbox/darwin-sandbox/6160/execroot/com_github_grpc_grpc/bazel-out/darwin-fastbuild/testlogs/src/objective-c/tests/UnitTests/test.outputs
    IDETestRunSpecificationPath = /var/folders/wv/_5q8c8j155g873yylt68nd4c0000gn/T/test_runner_work_dir.3ihwYP/TEST_ROOT/xctestrun.plist

Writing result bundle at path:
    /Volumes/BuildData/tmpfs/tmp/bazel/sandbox/darwin-sandbox/6160/execroot/com_github_grpc_grpc/bazel-out/darwin-fastbuild/testlogs/src/objective-c/tests/UnitTests/test.outputs/test.xcresult

Testing started on 'New-iPhone 11 Pro Max-13.3'
2022-05-17 15:56:08.497 xcodebuild[22549:56479]  IDETestOperationsObserverDebug: Writing diagnostic log for test session to:
/Volumes/BuildData/tmpfs/tmp/bazel/sandbox/darwin-sandbox/6160/execroot/com_github_grpc_grpc/bazel-out/darwin-fastbuild/testlogs/src/objective-c/tests/UnitTests/test.outputs/test.xcresult/Staging/1_Test/Diagnostics/UnitTests-F584CB4E-8DAD-4B8F-B20E-8FA2A9CDBFA2/UnitTests-CD0709C5-7B6A-45DB-834B-795A93B7B6E7/Session-UnitTests-2022-05-17_155608-GEBBKc.log
2022-05-17 15:56:08.497 xcodebuild[22549:56445] [MT] IDETestOperationsObserverDebug: (034A3956-8C82-4C3B-A375-96D7F35F5613) Beginning test session UnitTests-034A3956-8C82-4C3B-A375-96D7F35F5613 at 2022-05-17 15:56:08.497 with Xcode 11C29 on target <DVTiPhoneSimulator: 0x7ff4ec503ef0> {
        SimDevice: New-iPhone 11 Pro Max-13.3 (9C75AAC8-88CE-4003-9858-1C4A4D3C4BD4, iOS 13.3, Shutdown)
} (13.3 (17C45))
2022-05-17 15:56:56.108 xcodebuild[22549:56445] [MT] IDETestOperationsObserverDebug: (034A3956-8C82-4C3B-A375-96D7F35F5613) Finished requesting crash reports. Continuing with testing.
2022-05-17 15:58:37,937 The xcodebuild command got stuck and has not started test in 150. Will kill the command directly.

*** If you believe this error represents a bug, please attach the result bundle at /Volumes/BuildData/tmpfs/tmp/bazel/sandbox/darwin-sandbox/6160/execroot/com_github_grpc_grpc/bazel-out/darwin-fastbuild/testlogs/src/objective-c/tests/UnitTests/test.outputs/test.xcresult

** BUILD INTERRUPTED **
2022-05-17 15:58:52,561 xcodebuild command can not launch test on device/simulator in 150s.
Fatal error: Missing Info.plist: file /BuildRoot/Library/Caches/com.apple.xbs/Sources/XCResultKit/XCResultKit-15700/Sources/XCResultKit/RootInfoPlistParser.swift, line 52
2022-05-17 15:58:53,079
2022-05-17 15:58:53,080 Deleting simulator 9C75AAC8-88CE-4003-9858-1C4A4D3C4BD4 asynchronously.
2022-05-17 15:58:53,084 Done.
================================================================================
//src/objective-c/tests:UnitTests                                        FAILED in 168.5s

jtattermusch commented 2 years ago

Potentially relevant: https://github.com/bazelbuild/rules_apple/issues/185#issuecomment-500047365 https://github.com/bazel-ios/rules_ios/issues/234

grpc / grpc-ios

Switch to iOS Test Runner for all bazel-based test #94