parroty / excoveralls

Coverage report tool for Elixir with coveralls.io integration.
MIT License
820 stars 178 forks source link

Umbrella run ignores config json in root #313

Closed tpitale closed 1 year ago

tpitale commented 1 year ago

We recently updated to a new version of elixir (1.14.5) and our test coverage is now failing where it did not before (elixir 1.12).

Most of the apps under our umbrella do not have configured coverage. We have the minimum set in the root of the project at (say for example) 10%.

An individual app still outputs when running mix coveralls.html -u:

Generating cover results ...

Percentage | Module
-----------|--------------------------
     0.00% | SharedLibs.Clients.AuditEventClient
     0.00% | SharedLibs.Clients.AuditEventClient.AuditEvent
     0.00% | SharedLibs.ExpireAndInsert
     0.00% | SharedLibs.InsertAndUpdate
    71.43% | SharedLibs.HttpRetry
    78.57% | SharedLibs.Clients.AuthorizationClient
    86.81% | SharedLibs.Clients.NerdGraphClient
-----------|--------------------------
    49.53% | Total

Coverage test failed, threshold not met:

    Coverage:   49.53%
    Threshold:  90.00%

I've tried updating to the lates excoveralls.

tpitale commented 1 year ago

I'm trying to put configuration in the app/{appname}/coveralls.json for every app. It does not seem to be respected there either 🤔

tpitale commented 1 year ago

Some of these apps don't even have excoveralls configured in them. Maybe this is an issue with the umbrella lib?

tpitale commented 1 year ago

Seems like test_coverage: [tool: ExCoveralls] needs to be in every app in the umbrella for it to be included in the cumulative results. Testing out this theory.

tpitale commented 1 year ago

Okay … now everything is included in the summary. But now I can't ignore some of the apps. Even configuring skip_files in the root coveralls.json does not seem to skip them.

tpitale commented 1 year ago

I've now added coveralls.json to each app. Seems to be reaching the desired impact.