override option not working

[stuart-clark-exeter]

Hiya thanks for your work on this tool :)

I think I have found a bug with the override option...

This is my testing command:

gotestsum -- -covermode=atomic -coverprofile=coverage.out ./... && gocov convert coverage.out  | gocov-html > coverage-report.html && go-test-coverage --config=./.testcoverage.yml

If I run with the following .testcoverage.yml:

# (mandatory) 
# Path to coverprofile file (output of `go test -coverprofile` command)
profile: coverage.out

# Holds coverage thresholds percentages, values should be in range [0-100]
threshold:
  file: 90
  package: 90

# Holds regexp rules which will override thresholds for matched files or packages
# override:
#   # Increase coverage threshold to 100% for `foo` package (default is 80, as configured above)
#   # e.g.
#   # TODO this doesn't work
#   - threshold: 20
#     path: ^x/.*/types$

# Holds regexp rules which will exclude matched files or packages from coverage statistics
exclude:
  # Exclude files or packages matching their paths
  paths:
    - \.pb\.go$    # excludes all protobuf generated files

# NOTES:
# - symbol `/` in all path regexps will be replaced by
#   current OS file path separator to properly work on Windows

I get this output

DONE 100 tests in 70.524s
Took 2.090489875s
File coverage threshold (90%) satisfied:                                FAIL
  below threshold:                                                      coverage:       threshold:
  app/app.go                                                            4%              90%
  app/encoding.go                                                       0%              90%
  app/export.go                                                         0%              90%
  app/genesis.go                                                        0%              90%
  cmd/mimird/cmd/genaccounts.go                                         8%              90%
  cmd/mimird/cmd/root.go                                                41%             90%
  x/metahistory/client/cli/query.go                                     0%              90%
  x/metahistory/client/cli/query_meta_history.go                        88%             90%
  x/metahistory/client/cli/query_params.go                              0%              90%
  x/metahistory/client/cli/tx.go                                        0%              90%
  x/metahistory/keeper/keeper.go                                        75%             90%
  x/metahistory/keeper/msg_server.go                                    0%              90%
  x/metahistory/keeper/query_params.go                                  75%             90%
  x/metahistory/module.go                                               0%              90%
  x/metahistory/module_simulation.go                                    0%              90%
  x/metahistory/types/codec.go                                          0%              90%
  x/metahistory/types/keys.go                                           0%              90%
  x/metahistory/types/params.go                                         42%             90%
  x/metahistory/types/query.pb.gw.go                                    0%              90%
  x/organization/client/cli/query.go                                    0%              90%
  x/organization/client/cli/query_membership.go                         88%             90%
  x/organization/client/cli/query_mimir_org.go                          88%             90%
  x/organization/client/cli/query_params.go                             0%              90%
  x/organization/client/cli/tx.go                                       0%              90%
  x/organization/client/cli/tx_create_identity.go                       0%              90%
  x/organization/keeper/identity.go                                     63%             90%
  x/organization/keeper/keeper.go                                       75%             90%
  x/organization/keeper/query_params.go                                 75%             90%
  x/organization/module.go                                              0%              90%
  x/organization/module_simulation.go                                   0%              90%
  x/organization/types/codec.go                                         0%              90%
  x/organization/types/genesis.go                                       88%             90%
  x/organization/types/key_organization.go                              0%              90%
  x/organization/types/keys.go                                          0%              90%
  x/organization/types/message_create_identity.go                       30%             90%
  x/organization/types/params.go                                        53%             90%
  x/organization/types/query.pb.gw.go                                   0%              90%
  x/organization/types/role.go                                          0%              90%
  x/transcripts/keeper/keeper.go                                        75%             90%
  x/transcripts/keeper/msg_server.go                                    0%              90%
  x/transcripts/keeper/msg_server_create_transcript.go                  0%              90%
  x/transcripts/keeper/msg_server_create_transcript_collection.go       0%              90%
  x/transcripts/keeper/query_params.go                                  75%             90%
  x/transcripts/module.go                                               0%              90%
  x/transcripts/module_simulation.go                                    0%              90%
  x/transcripts/types/codec.go                                          0%              90%
  x/transcripts/types/keys.go                                           0%              90%
  x/transcripts/types/message_create_transcript.go                      30%             90%
  x/transcripts/types/message_create_transcript_collection.go           30%             90%
  x/transcripts/types/params.go                                         42%             90%
  x/transcripts/types/query.pb.gw.go                                    0%              90%
Package coverage threshold (90%) satisfied:     FAIL
  below threshold:                              coverage:       threshold:
  app                                           2%              90%
  x/metahistory/keeper                          87%             90%
  x/metahistory/types                           6%              90%
  x/organization/keeper                         89%             90%
  x/organization/types                          8%              90%
  x/transcripts/keeper                          36%             90%
  x/transcripts/types                           13%             90%
  cmd/mimird/cmd                                29%             90%
  x/metahistory/client/cli                      55%             90%
  x/metahistory                                 20%             90%
  x/organization/client/cli                     55%             90%
  x/organization                                30%             90%
  x/transcripts                                 8%              90%
Total coverage threshold (0%) satisfied:        PASS
Total test coverage: 28%
Warning: run-commands command "gotestsum -- -covermode=atomic -coverprofile=coverage.out ./... && gocov convert coverage.out  | gocov-html > coverage-report.html && go-test-coverage --config=./.testcoverage.yml" exited with non-zero status code

I want to override the types directories so I am using this to do so (note I have uncommented the overrides key):

# (mandatory) 
# Path to coverprofile file (output of `go test -coverprofile` command)
profile: coverage.out

# Holds coverage thresholds percentages, values should be in range [0-100]
threshold:
  file: 90
  package: 90

# Holds regexp rules which will override thresholds for matched files or packages
# override:
  # Increase coverage threshold to 100% for `foo` package (default is 80, as configured above)
  # e.g.
  # TODO this doesn't work
  - threshold: 20
    path: ^x/.*/types$

# Holds regexp rules which will exclude matched files or packages from coverage statistics
exclude:
  # Exclude files or packages matching their paths
  paths:
    - \.pb\.go$    # excludes all protobuf generated files

# NOTES:
# - symbol `/` in all path regexps will be replaced by
#   current OS file path separator to properly work on Windows

This gives me the following output

File coverage threshold (90%) satisfied:                                FAIL
  below threshold:                                                      coverage:       threshold:
  app/app.go                                                            4%              20%
  app/encoding.go                                                       0%              20%
  app/export.go                                                         0%              20%
  app/genesis.go                                                        0%              20%
  cmd/mimird/cmd/genaccounts.go                                         8%              20%
  x/metahistory/client/cli/query.go                                     0%              20%
  x/metahistory/client/cli/query_params.go                              0%              20%
  x/metahistory/client/cli/tx.go                                        0%              20%
  x/metahistory/keeper/msg_server.go                                    0%              20%
  x/metahistory/module.go                                               0%              20%
  x/metahistory/module_simulation.go                                    0%              20%
  x/metahistory/types/codec.go                                          0%              20%
  x/metahistory/types/keys.go                                           0%              20%
  x/metahistory/types/query.pb.gw.go                                    0%              20%
  x/organization/client/cli/query.go                                    0%              20%
  x/organization/client/cli/query_params.go                             0%              20%
  x/organization/client/cli/tx.go                                       0%              20%
  x/organization/client/cli/tx_create_identity.go                       0%              20%
  x/organization/module.go                                              0%              20%
  x/organization/module_simulation.go                                   0%              20%
  x/organization/types/codec.go                                         0%              20%
  x/organization/types/key_organization.go                              0%              20%
  x/organization/types/keys.go                                          0%              20%
  x/organization/types/query.pb.gw.go                                   0%              20%
  x/organization/types/role.go                                          0%              20%
  x/transcripts/keeper/msg_server.go                                    0%              20%
  x/transcripts/keeper/msg_server_create_transcript.go                  0%              20%
  x/transcripts/keeper/msg_server_create_transcript_collection.go       0%              20%
  x/transcripts/module.go                                               0%              20%
  x/transcripts/module_simulation.go                                    0%              20%
  x/transcripts/types/codec.go                                          0%              20%
  x/transcripts/types/keys.go                                           0%              20%
  x/transcripts/types/query.pb.gw.go                                    0%              20%

Package coverage threshold (90%) satisfied:     FAIL
  below threshold:                              coverage:       threshold:
  x/transcripts/types                           13%             20%
  app                                           2%              20%
  x/metahistory/types                           6%              20%
  x/organization/types                          8%              20%
  x/transcripts                                 8%              20%

Total coverage threshold (0%) satisfied:        PASS
Total test coverage: 28%

As you can see the threshold has been set to 20% for all files not just the ones that match the path given.

Again thanks for all the work, this is otherwise just what I have been looking for!

[stuart-clark-exeter]

Just found that changing the path to ^x/.*/types/.*\.go$ resolved the issue, a bit of bad regex by me :)

I guess tho that there is still a minor issue here as the threshold output is still wrong I think?

[vladopajic]

Just found that changing the path to ^x/./types/..go$ resolved the issue, a bit of bad regex by me :)

Regexp should be changed to what you are suggesting or ^x/.*/types/ (without $ sign at the end).

I guess tho that there is still a minor issue here as the threshold output is still wrong I think?

Yes, it appears so...

Thanks for reporting :)

[vladopajic]

@stuart-clark-exeter this should be fixed in v2.7.1, could you check if this issue is resolved?

[stuart-clark-exeter]

Thanks for getting back so fast @vladopajic in the end I found I had to use ^x/.*/types(/|$)

and everything seems to be working fine now :)