search

irongut / CodeCoverageSummary

A GitHub Action that reads Cobertura format code coverage files and outputs a text or markdown summary.
MIT License
187 stars 53 forks source link

Code Coverage incorrectly being calculated for solution with multiple Tests projects and shared libraries #318

Open fbridger opened 1 month ago

fbridger commented 1 month ago

Bug Report

When running CCS on a solution that has multiple test projects that shares multiple libraries (common libraries) it generates multiple code coverage files.

Visual Studio Code Coverage generates the calculation correctly which generates a line coverage of 80,27%

However, CCS is calculating a line coverage of 43%. It seems CCS is not calculating the code coverage correctly for projects that are shared as they sometime have 0% coverage when they are run in other Tests projects.

Example:

Coverage File: /github/workspace/coverage/08136505-8dcb-4358-9aeb-6c4c4593270a/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/cef530ca-79b4-480f-846c-475814d8b0b2/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/56c851f9-eb73-4de4-a39a-2c7a44bb88d1/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/7f265646-44cc-4a47-9eb2-3f87de515b25/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/a16ba91b-0337-4b63-9ee9-4a0a1891ec6c/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/3f139be9-5ce4-4548-990d-503e5b01882a/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/5c94f607-98e1-40ac-9613-d88dea525dee/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/a6f9626d-3b2f-449f-89b9-3ca3261ff43d/coverage.cobertura.xml
Coverage File: /github/workspace/coverage/194e7495-7f8c-4d06-b7c1-dd5d31aa9ad6/coverage.cobertura.xml

![Code Coverage](https://img.shields.io/badge/Code%20Coverage-43%25-critical?style=flat)

Package | Line Rate | Branch Rate | Health
-------- | --------- | ----------- | ------
Acme.Common.Aws | 95% | 100% | ✔
Acme.Common.Domain | 0% | 100% | ❌
Acme.Weather.Providers | 100% | 100% | ✔
Acme.Weather.Domain | 21% | 0% | ❌
Acme.Weather.Providers.AWS | 12% | 0% | ❌
Acme.Common.Aws | 0% | 0% | ❌
Acme.Common.Caching | 0% | 0% | ❌
Acme.Weather.Services | 97% | 81% | ✔
Acme.Common.Ftp | 0% | 0% | ❌
Acme.Weather.Providers.Http | 11% | 0% | ❌
Acme.Common.Domain | 0% | 100% | ❌
Acme.Weather.Providers.Azure | 0% | 100% | ❌
Acme.Weather.Providers.Ftp | 11% | 0% | ❌
Acme.Weather.Providers | 100% | 100% | ✔
Acme.Weather.Domain | 91% | 78% | ✔
Acme.Common.Api | 0% | 0% | ❌
Acme.Weather.Providers.AWS | 0% | 0% | ❌
Acme.Common.Aws | 0% | 0% | ❌
Acme.Weather.Services.Smart | 0% | 0% | ❌
Acme.Common.Caching | 0% | 0% | ❌
Acme.Weather.Services | 0% | 0% | ❌
Acme.Common.Ftp | 0% | 0% | ❌
Acme.Weather.Providers.Http | 0% | 0% | ❌
Acme.Common.Domain | 55% | 100% | ❌
Acme.Weather.Api | 80% | 63% | ➖
Acme.Weather.Providers.Azure | 0% | 100% | ❌
Acme.Weather.Providers.Ftp | 0% | 0% | ❌
Acme.Weather.Providers | 100% | 100% | ✔
Acme.Weather.Domain | 16% | 0% | ❌
Acme.Weather.Providers.AWS | 100% | 86% | ✔
Acme.Common.Aws | 0% | 0% | ❌
Acme.Common.Domain | 0% | 100% | ❌
Acme.Common.Api | 0% | 0% | ❌
Acme.Common.Domain | 0% | 100% | ❌
Acme.Weather.Providers | 100% | 100% | ✔
Acme.Weather.Domain | 22% | 0% | ❌
Acme.Weather.Providers.AWS | 0% | 0% | ❌
Acme.Common.Aws | 0% | 0% | ❌
Acme.Weather.Services.Smart | 92% | 91% | ✔
Acme.Common.Caching | 0% | 0% | ❌
Acme.Weather.Services | 0% | 0% | ❌
Acme.Common.Ftp | 0% | 0% | ❌
Acme.Weather.Providers.Http | 0% | 0% | ❌
Acme.Common.Domain | 0% | 100% | ❌
Acme.Weather.Providers.Azure | 0% | 100% | ❌
Acme.Weather.Providers.Ftp | 0% | 0% | ❌
Acme.Common.Repositories.Mongo | 75% | 36% | ➖
Acme.Common.Domain | 15% | 100% | ❌
Acme.Orders.Api | 93% | 62% | ✔
Acme.Common.Api | 50% | 12% | ❌
Acme.Shop.Domain | 100% | 100% | ✔
Acme.Common.Caching | 0% | 0% | ❌
Acme.Shop.Repositories | 100% | 100% | ✔
Acme.Common.Domain | 35% | 100% | ❌
Acme.Shop.Services | 0% | 100% | ❌
Acme.Common.Caching | 82% | 0% | ✔
Acme.Common.Domain | 30% | 100% | ❌
**Summary** | **43%** (1175 / 2936) | **33%** (252 / 766) | ❌

_Minimum allowed line rate is `60%`_

FAIL: Overall line rate below minimum threshold of 60%.

Linked To

https://github.com/irongut/CodeCoverageSummary/issues/38 https://github.com/irongut/CodeCoverageSummary/issues/19

CFarinellaCM commented 3 weeks ago

I encountered the same issue and tried the following which seems to be a workaround.

Try using reportgenerator to merge the reports:

      - name: Merge Coverage Reports
        run: reportgenerator -reports:./coverage/**/coverage.cobertura.xml -targetdir:./coverage -reporttypes:Cobertura

Then reference the merged filename instead of the wild card

- name: Code Coverage Report
        uses: irongut/CodeCoverageSummary@v1.3.0
        with:
          filename: ./coverage/Cobertura.xml