Code coverage is lost for first multi_scan run

john-mejia commented 3 years ago

New Issue Checklist

[x] Updated fastlane-plugin-test_center to the latest version
[x] I read the README.md
[x] I reviewed the example(s) for the action(s) I am using
[x] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see
[x] I have reviewed the Discussions forum to see my question has already been addressed.

If you love this fastlane plugin, consider sponsoring it or asking your company to sponsor it. I would really appreciate any gesture: https://github.com/sponsors/lyndsey-ferguson. 😍

✅ fastlane environment ✅

### Stack

| Key                         | Value                                                    |
| --------------------------- | -------------------------------------------------------- |
| OS                          | 11.2.1                                                   |
| Ruby                        | 2.6.1                                                    |
| Bundler?                    | true                                                     |
| Git                         | git version 2.24.3 (Apple Git-128)                       |
| Installation Source         | ~/.rbenv/versions/2.6.1/lib/ruby/gems/2.6.0/bin/fastlane |
| Host                        | macOS 11.2.1 (20D74)                                     |
| Ruby Lib Dir                | ~/.rbenv/versions/2.6.1/lib                              |
| OpenSSL Version             | OpenSSL 1.0.2q  20 Nov 2018                              |
| Is contained                | false                                                    |
| Is homebrew                 | false                                                    |
| Is installed via Fabric.app | false                                                    |
| Xcode Path                  | /Applications/Xcode.app/Contents/Developer/              |
| Xcode Version               | 12.4                                                     |

### System Locale

| Variable | Value       |   |
| -------- | ----------- | - |
| LANG     | en_US.UTF-8 | ✅ |
| LC_ALL   |             |   |
| LANGUAGE |             |   |

### fastlane files:

`./fastlane/Fastfile`

```ruby
# This file contains the fastlane.tools configuration
# You can find the documentation at https://docs.fastlane.tools
#
# For a list of all available actions, check out
#
#     https://docs.fastlane.tools/actions
#
# For a list of all available plugins, check out
#
#     https://docs.fastlane.tools/plugins/available-plugins
#

# Uncomment the line if you want fastlane to automatically update itself
# update_fastlane

default_platform(:ios)

platform :ios do

  desc 'Run tests (and re-run flakes)'
  lane :test do
    test_run_block = lambda do |testrun_info|
      failed_test_count = testrun_info[:failed].size

      if failed_test_count > 0
        UI.important('The run of tests would finish with failures due to fragile tests here.')

        try_attempt = testrun_info[:try_count]
        if try_attempt < 2
          UI.header('Since we are using :multi_scan, we can re-run just those failing tests!')
        end
      end
    end

    result = multi_scan(
      workspace: File.absolute_path('../testCoverage.xcworkspace'),
      try_count: 1,
      fail_build: true,
      include_simulator_logs: false,
      scheme: 'testCoverage',
      testrun_completed_block: test_run_block,
      code_coverage: true,
    )
    unless result[:failed_testcount].zero?
      UI.message("There are #{result[:failed_testcount]} legitimate failing tests")
    end
  end

end

```


`./fastlane/Appfile`

```ruby
# app_identifier("[[APP_IDENTIFIER]]") # The bundle identifier of your app
# apple_id("[[APPLE_ID]]") # Your Apple email address

# For more information about the Appfile, see:
#     https://docs.fastlane.tools/advanced/#appfile

```


### fastlane gems

| Gem      | Version | Update-Status |
| -------- | ------- | ------------- |
| fastlane | 2.176.0 | ✅ Up-To-Date  |

### Loaded fastlane plugins:

| Plugin                      | Version | Update-Status |
| --------------------------- | ------- | ------------- |
| fastlane-plugin-test_center | 3.15.3  | ✅ Up-To-Date  |

Loaded gems

| Gem                           | Version      |
| ----------------------------- | ------------ |
| did_you_mean                  | 1.3.0        |
| bundler                       | 2.2.7        |
| rake                          | 13.0.3       |
| CFPropertyList                | 3.0.3        |
| public_suffix                 | 4.0.6        |
| addressable                   | 2.7.0        |
| artifactory                   | 3.0.15       |
| atomos                        | 0.1.3        |
| aws-eventstream               | 1.1.0        |
| aws-partitions                | 1.429.0      |
| aws-sigv4                     | 1.2.2        |
| jmespath                      | 1.4.0        |
| aws-sdk-core                  | 3.112.0      |
| aws-sdk-kms                   | 1.42.0       |
| aws-sdk-s3                    | 1.88.1       |
| babosa                        | 1.0.4        |
| claide                        | 1.0.3        |
| colored                       | 1.2          |
| colored2                      | 3.1.2        |
| colorize                      | 0.8.1        |
| highline                      | 1.7.10       |
| commander-fastlane            | 4.4.6        |
| declarative                   | 0.0.20       |
| declarative-option            | 0.1.0        |
| digest-crc                    | 0.6.3        |
| unf_ext                       | 0.0.7.7      |
| unf                           | 0.1.4        |
| domain_name                   | 0.5.20190701 |
| dotenv                        | 2.7.6        |
| emoji_regex                   | 3.2.2        |
| excon                         | 0.79.0       |
| faraday-net_http              | 1.0.1        |
| multipart-post                | 2.0.0        |
| ruby2_keywords                | 0.0.4        |
| faraday                       | 1.3.0        |
| http-cookie                   | 1.0.3        |
| faraday-cookie_jar            | 0.0.7        |
| faraday_middleware            | 1.0.0        |
| fastimage                     | 2.2.2        |
| gh_inspector                  | 1.1.3        |
| jwt                           | 2.2.2        |
| memoist                       | 0.16.2       |
| multi_json                    | 1.15.0       |
| os                            | 1.1.1        |
| signet                        | 0.14.1       |
| googleauth                    | 0.15.1       |
| httpclient                    | 2.8.3        |
| mini_mime                     | 1.0.2        |
| uber                          | 0.1.0        |
| representable                 | 3.0.4        |
| retriable                     | 3.1.2        |
| google-api-client             | 0.38.0       |
| rexml                         | 3.2.4        |
| webrick                       | 1.7.0        |
| google-apis-core              | 0.2.1        |
| google-apis-iamcredentials_v1 | 0.1.0        |
| google-apis-storage_v1        | 0.2.0        |
| google-cloud-env              | 1.4.0        |
| google-cloud-errors           | 1.0.1        |
| google-cloud-core             | 1.5.0        |
| google-cloud-storage          | 1.30.0       |
| json                          | 2.5.1        |
| mini_magick                   | 4.11.0       |
| naturally                     | 2.2.1        |
| plist                         | 3.6.0        |
| rubyzip                       | 2.3.0        |
| security                      | 0.1.3        |
| simctl                        | 1.6.8        |
| slack-notifier                | 2.3.2        |
| terminal-notifier             | 2.0.0        |
| unicode-display_width         | 1.7.0        |
| terminal-table                | 1.8.0        |
| tty-screen                    | 0.8.1        |
| tty-cursor                    | 0.7.1        |
| tty-spinner                   | 0.9.3        |
| word_wrap                     | 1.0.0        |
| nanaimo                       | 0.3.0        |
| xcodeproj                     | 1.19.0       |
| rouge                         | 2.0.7        |
| xcpretty                      | 0.3.0        |
| xcpretty-travis-formatter     | 1.0.1        |
| trainer                       | 0.9.1        |
| xctest_list                   | 1.2.1        |
| fastlane-plugin-test_center   | 3.15.3       |


*generated on:* **2021-02-25**

Patrick-Kladek commented 3 years ago

I have the same issue. Is there a known workaround?

lyndsey-ferguson commented 3 years ago

Hi @Patrick-Kladek , I'm not aware of any workaround yet. However, you may want to use the testrun_completed_block to pass in a code block that finds and moves the code coverage file out and then add some code in your fastlane to combine them into one code coverage file.

Here is an example of how to use the block: https://github.com/lyndsey-ferguson/fastlane-plugin-test_center/blob/master/docs/feature_details/multi_scan.md#examples

lyndsey-ferguson commented 2 years ago

I want to apologize that I have not responded to this issue. A family health issue is requiring me to narrow my focus on the essentials and I don't have the time I need to focus on the plugin more than a minute. If you can get some other contributors to this project to help with this, I would consider merging a resulting PR.

lyndsey-ferguson / fastlane-plugin-test_center

Code coverage is lost for first multi_scan run #330

New Issue Checklist