fastlane multi_scan: merge xcresult files error

louisssliao commented 2 years ago

New Issue Checklist

[x] Updated fastlane-plugin-test_center to the latest version
[x] I read the README.md
[x] I reviewed the example(s) for the action(s) I am using
[x] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
[x] I have reviewed the Discussions forum to see my question has already been addressed.

Plugin version

Fastlane: 2.209.0 fastlane-plugin-test_center: 3.19.1

Issue Description

I upgraded to Fastlane 2.209.0 and run multi_scan, it got an error after finishing the tests, looks like it got an error when merging xcresult files. This error doesn't happen in Fastlane 2.200.0.

Complete output when running fastlane, including the stack trace and command used

multi_scan(
            workspace: "XXXXX",
            scheme: "XXXXX",
            testplan: "XXXXX",
            device: "iPhone 12 (15.5)",
            derived_data_path: "${HOME}/Derived_Data/Instance_UI_Test",
            buildlog_path: "${HOME}/Derived_Data/Instance_UI_Test/scan",
            output_types: 'junit',
            result_bundle: true,
            try_count: 1
            parallel_testrun_count: 2,
            retry_test_runner_failures: true,
            clean: true,
            reinstall_app: true,
            app_identifier: "XXXXX",
            simulator_started_callback: sim_callback,
            skip_slack: true,
            )

After finishing the test, it shows an error.

[worker 2] [15:00:45]: ▸ Selected tests
[worker 2] [15:00:45]: ▸ GrindrUITests.xctest
[worker 2] [15:00:45]: ▸ GrindrUITestsCascade
[worker 2] [15:02:45]: ▸     ✔ testFreeUserGets100Cells (119.540 seconds)
[worker 2] [15:02:45]: ▸ 2022-08-22 15:02:45.305 xcodebuild[36897:32662703] [MT] IDETestOperationsObserverDebug: 140.822 elapsed -- Testing started completed.
[worker 2] [15:02:45]: ▸ 2022-08-22 15:02:45.305 xcodebuild[36897:32662703] [MT] IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
[worker 2] [15:02:45]: ▸ 2022-08-22 15:02:45.305 xcodebuild[36897:32662703] [MT] IDETestOperationsObserverDebug: 140.822 sec, +140.822 sec -- end
[worker 2] [15:02:47]: ▸ Test Execute Succeeded
[worker 2] [15:02:47]:
[15:03:13]: $ xcrun xcresulttool merge /Users/dev/code/grindr/grindr-3.0-ios/fastlane/test_output/GrindrUITests-batch-2/report.xcresult /Users/dev/code/grindr/grindr-3.0-ios/fastlane/test_output/GrindrUITests-batch-1/report.xcresult --output-path /var/folders/vs/dskdwrjd0qz_5pshbzllg71r0000gn/T/collated_result_20220822-25319-old3t.xcresult
[15:03:14]: ▸ [v3] Merged to: /var/folders/vs/dskdwrjd0qz_5pshbzllg71r0000gn/T/collated_result_20220822-25319-old3t.xcresult
[15:03:14]: Error: No such file or directory @ dir_s_mkdir - /Users/dev/code/grindr/grindr-3.0-ios/fastlane/test_output/GrindrUITests/report.xcresult

lyndsey-ferguson commented 2 years ago

Could you attach a sample project to facilitate a fix?

louisssliao commented 2 years ago

Hello @lyndsey-ferguson, really sorry for the late reply, I was extremely busy on other projects. Here is the sample repository that can reproduce this symptom. I upgraded to Xcode14, iOS16, Fastlane 2.210.0, and TestCenter 3.19.1 and still can reproduce it. https://github.com/louisssliao/SampleApp-TestCenter-Error

Here are the steps of using this repo:

bundle install
sh run_local_UITests.sh

lyndsey-ferguson / fastlane-plugin-test_center