Feature Request: show passed test cases from failed suites in pass section in html

jineshqa commented 5 years ago

New Issue Checklist

[X] Updated fastlane-plugin-test_center to the latest version
[X] I read the README.md
[X] I reviewed the example(s) for the action(s) I am using
[X] I have removed any sensitive data such as passwords, authentication tokens, or anything else I do not want to world to see

Issue Description

I am using multi_scan which generates HTML report at the end. I observed that pass section is showing only passed test suites. Passing test cases from failing test suites are not displayed at all.

after clicking passing does not show _all_ passes

Complete output when running fastlane, including the stack trace and command used

NA

Environment

✅ fastlane environment ✅

### Stack

| Key                         | Value                                       |
| --------------------------- | ------------------------------------------- |
| OS                          | 10.14.6                                     |
| Ruby                        | 2.6.3                                       |
| Bundler?                    | false                                       |
| Git                         | git version 2.20.1 (Apple Git-117)          |
| Installation Source         | ~/.gems/bin/fastlane                        |
| Host                        | Mac OS X 10.14.6 (18G103)                   |
| Ruby Lib Dir                | ~/.rbenv/versions/2.6.3/lib                 |
| OpenSSL Version             | OpenSSL 1.1.1d  10 Sep 2019                 |
| Is contained                | false                                       |
| Is homebrew                 | false                                       |
| Is installed via Fabric.app | false                                       |
| Xcode Path                  | /Applications/Xcode.app/Contents/Developer/ |
| Xcode Version               | 10.2.1                                      |

### System Locale

| Variable | Value       |   |
| -------- | ----------- | - |
| LANG     | en_US.UTF-8 | ✅ |
| LC_ALL   |             |   |
| LANGUAGE |             |   |

### fastlane files:

`./fastlane/Fastfile`

```ruby
# This file contains the fastlane.tools configuration
# You can find the documentation at https://docs.fastlane.tools
#
# For a list of all available actions, check out
#
#     https://docs.fastlane.tools/actions
#
# For a list of all available plugins, check out
#
#     https://docs.fastlane.tools/plugins/available-plugins
#

# Uncomment the line if you want fastlane to automatically update itself
# update_fastlane

default_platform(:ios)

platform :ios do
  lane :sweep do
    begin
      sh 'rm -rf ./test_output'
    rescue
    end

    test_run_block = lambda do |testrun_info|
      failed_test_count = testrun_info[:failed].size

      if failed_test_count > 0
        UI.important('The run of tests would finish with failures due to fragile tests here.')

        try_attempt = testrun_info[:try_count]
        if try_attempt < 3
          UI.header('Since we are using :multi_scan, we can re-run just those failing tests!')
        end
      end
    end

    result = multi_scan(
      try_count: 3,
      fail_build: false,
      scheme: 'collate_errors',
      result_bundle: true,
      device: "iPhone 11 (13.1)",
      testrun_completed_block: test_run_block,
      output_directory: './fastlane/test_output',
      output_types: 'junit,html',
      buildlog_path: './fastlane/test_output/',
      code_coverage: true,
      invocation_based_tests: false,
      parallel_testrun_count: 2
    )
    unless result[:failed_testcount].zero?
      UI.message("There are #{result[:failed_testcount]} legitimate failing tests")
    end
  end
end

```


`./fastlane/Appfile`

```ruby
# app_identifier("[[APP_IDENTIFIER]]") # The bundle identifier of your app
# apple_id("[[APPLE_ID]]") # Your Apple email address

# For more information about the Appfile, see:
#     https://docs.fastlane.tools/advanced/#appfile

```


### fastlane gems

| Gem      | Version | Update-Status |
| -------- | ------- | ------------- |
| fastlane | 2.134.0 | ✅ Up-To-Date  |

### Loaded fastlane plugins:

| Plugin                      | Version | Update-Status |
| --------------------------- | ------- | ------------- |
| fastlane-plugin-test_center | 3.8.10  | ✅ Up-To-Date  |

Loaded gems

| Gem                         | Version      |
| --------------------------- | ------------ |
| did_you_mean                | 1.3.0        |
| slack-notifier              | 2.3.2        |
| atomos                      | 0.1.3        |
| CFPropertyList              | 3.0.1        |
| claide                      | 1.0.3        |
| colored2                    | 3.1.2        |
| nanaimo                     | 0.2.6        |
| xcodeproj                   | 1.13.0       |
| rouge                       | 2.0.7        |
| xcpretty                    | 0.3.0        |
| terminal-notifier           | 2.0.0        |
| terminal-table              | 1.8.0        |
| plist                       | 3.5.0        |
| addressable                 | 2.7.0        |
| multipart-post              | 2.0.0        |
| word_wrap                   | 1.0.0        |
| public_suffix               | 2.0.5        |
| tty-screen                  | 0.7.0        |
| tty-cursor                  | 0.7.0        |
| tty-spinner                 | 0.9.1        |
| babosa                      | 1.0.3        |
| colored                     | 1.2          |
| highline                    | 1.7.10       |
| commander-fastlane          | 4.4.6        |
| excon                       | 0.67.0       |
| faraday                     | 0.17.0       |
| unf_ext                     | 0.0.7.6      |
| unf                         | 0.1.4        |
| domain_name                 | 0.5.20190701 |
| http-cookie                 | 1.0.3        |
| faraday-cookie_jar          | 0.0.6        |
| faraday_middleware          | 0.13.1       |
| fastimage                   | 2.1.7        |
| gh_inspector                | 1.1.3        |
| mini_magick                 | 4.9.5        |
| multi_xml                   | 0.6.0        |
| rubyzip                     | 1.3.0        |
| security                    | 0.1.3        |
| xcpretty-travis-formatter   | 1.0.0        |
| dotenv                      | 2.7.5        |
| bundler                     | 2.0.2        |
| naturally                   | 2.2.0        |
| simctl                      | 1.6.6        |
| jwt                         | 2.1.0        |
| uber                        | 0.1.0        |
| declarative                 | 0.0.10       |
| declarative-option          | 0.1.0        |
| representable               | 3.0.4        |
| retriable                   | 3.1.2        |
| mime-types-data             | 3.2019.1009  |
| mime-types                  | 3.3          |
| multi_json                  | 1.14.1       |
| signet                      | 0.12.0       |
| memoist                     | 0.16.0       |
| os                          | 1.0.1        |
| googleauth                  | 0.6.7        |
| httpclient                  | 2.8.3        |
| google-api-client           | 0.23.9       |
| google-cloud-env            | 1.2.1        |
| google-cloud-core           | 1.3.2        |
| digest-crc                  | 0.4.1        |
| google-cloud-storage        | 1.16.0       |
| emoji_regex                 | 1.0.1        |
| unicode-display_width       | 1.6.0        |
| json                        | 2.2.0        |
| forwardable                 | 1.2.0        |
| logger                      | 1.3.0        |
| stringio                    | 0.0.2        |
| ipaddr                      | 1.2.2        |
| openssl                     | 2.1.2        |
| ostruct                     | 0.1.0        |
| strscan                     | 1.0.0        |
| date                        | 2.0.0        |
| fileutils                   | 1.1.0        |
| etc                         | 1.0.1        |
| io-console                  | 0.4.7        |
| zlib                        | 1.0.0        |
| mini_portile2               | 2.4.0        |
| nokogiri                    | 1.10.4       |
| rexml                       | 3.1.9        |
| psych                       | 3.1.0        |
| mutex_m                     | 0.1.0        |
| xctest_list                 | 1.1.8        |
| colorize                    | 0.8.1        |
| fastlane-plugin-test_center | 3.8.10       |


*generated on:* **2019-10-21**

Attaching my report.html

report.html.zip

lyndsey-ferguson commented 5 years ago

@jineshqa and this works in normal scan? As in the passing tests from a failing testsuite show up?

jineshqa commented 5 years ago

Stopped using normal scan once I started using this :-) This is not a blocker for me so I'll try to reproduce this using scan only later today or perhaps tomorrow.

lyndsey-ferguson commented 5 years ago

I'm just curious if this ever worked. I am not changing the CSS from what it should be based on my understanding of what is there.

lyndsey-ferguson commented 5 years ago

I'm going to make this a feature request.

lyndsey-ferguson commented 2 years ago

I want to apologize that I have not responded to this issue. A family health issue is requiring me to narrow my focus on the essentials and I don't have the time I need to focus on the plugin more than a minute. If you can get some other contributors to this project to help with this, I would consider merging a resulting PR.

lyndsey-ferguson / fastlane-plugin-test_center