fastlane-community / trainer

Convert xcodebuild plist and xcresult files to JUnit reports
https://krausefx.com
MIT License
249 stars 50 forks source link

can't find results with Xcode 11 #39

Open bitcoder opened 5 years ago

bitcoder commented 5 years ago

Hi, i'm running "fastlane test" in a simple example project and using the "trainer" plugin to generate JUnit XML report. However, whenever I run fastlane it is not able of finding the results.

This is my "fastlane/Fastfile"

default_platform(:ios)

platform :ios do
  desc "Run tests"
  lane :test do
    scan(scheme: "UnitTest-Calculator",
       output_types: "",
       fail_build: false)

    trainer(output_directory: "build/reports/")
  end

end

This is the output obtained:

+----------------------------+-------------------------------------------------------------------------------------------+
|                                                      Lane Context                                                      |
+----------------------------+-------------------------------------------------------------------------------------------+
| DEFAULT_PLATFORM           | ios                                                                                       |
| PLATFORM_NAME              | ios                                                                                       |
| LANE_NAME                  | ios test                                                                                  |
| SCAN_DERIVED_DATA_PATH     | /Users/smsf/Library/Developer/Xcode/DerivedData/UnitTest-Calculator-fbcsrzqmxnbtunckthqv  |
|                            | azzqktol                                                                                  |
# This file contains the fastlane.tools configuration
| SCAN_GENERATED_PLIST_FILES | []                                                                                        |
| SCAN_GENERATED_PLIST_FILE  |                                                                                           |
+----------------------------+-------------------------------------------------------------------------------------------+
[12:10:06]: No test result files found in directory '.', make sure the file name ends with 'TestSummaries.plist' or '.xcresult'

+------+------------------+-------------+
|           fastlane summary            |
+------+------------------+-------------+
| Step | Action           | Time (in s) |
+------+------------------+-------------+
| 1    | default_platform | 0           |
| 2    | scan             | 23          |
| 💥   | trainer          | 0           |
+------+------------------+-------------+

[12:10:06]: fastlane finished with errors

[!] No test result files found in directory '.', make sure the file name ends with 'TestSummaries.plist' or '.xcresult'

This is the content of the build directory, under the "DerivedData" default one:

$ ls /Users/smsf/Library/Developer/Xcode/DerivedData/UnitTest-Calculator-fbcsrzqmxnbtunckthqvazzqktol/Logs/Test/
LogStoreManifest.plist                                     Run-UnitTest-Calculator-2019.10.24_12-05-37-+0100.xcresult

The "Run-UnitTest-Calculator-2019.10.24_12-05-37-+0100.xcresult" is not a file but a directory instead. It does not contain any TestSummaries.plist neither a .xcresult file.

Am I missing something basic in the configuration of fastlane or in the build itself to generate .xresult file? As it seems, XCode 11 is generating a .xcresult directory with many files. From the trainer code, from my understanding it does not go into .xcresult directories under the DerivedData folder.. but anyway, I don't see a specific .xcresult there.

dheerajSingla0 commented 4 years ago

I am facing the same issue. Have you found any work around or any solution?

bitcoder commented 4 years ago

Unfortunately the only solution was to use another tool: xcpretty. You can see an example here: https://confluence.xpand-it.com/display/XRAY/Testing+iOS+apps+using+XCTest+in+Swift

pyby commented 4 years ago

We have the same issue with Xcode 11.2 and Mojave 10.14.6.

The current workaround we applied is to copy the last xcresult bundle to the fastlane folder (or project folder to not add path to trainer action…).

trainer parses xcresult bundles. It's ok for that.

Xcode only keep the latest 2 xcresult bundles. If "latest bundle" is not safe enough, for sure, trainer has to check LogStoreManifest.plist.

zanizrules commented 4 years ago

Unfortunately the only solution was to use another tool: xcpretty. You can see an example here: https://confluence.xpand-it.com/display/XRAY/Testing+iOS+apps+using+XCTest+in+Swift

Issue I found after switching to rely on xcpretty is it can only handle basic use cases. For me I needed it to work with multiple devices so I put together this plugin

ghost commented 4 years ago

@bitcoder The following configuration worked for me. The main config is result_bundle: true trainer(path: "./fastlane/test_output", output_directory: "./fastlane")

default_platform(:ios)

  platform :ios do
  desc "Run tests"
  lane :test do
    scan(scheme: "UnitTest-Calculator",
      output_types: "",
      fail_build: false,
      result_bundle: true
    )
    trainer(path: "./fastlane/test_output", output_directory: "./fastlane")
  end
end