search

danielpalme / ReportGenerator

ReportGenerator converts coverage reports generated by coverlet, OpenCover, dotCover, Visual Studio, NCover, Cobertura, JaCoCo, Clover, gcov or lcov into human readable reports in various formats.
https://reportgenerator.io
Apache License 2.0
2.65k stars 283 forks source link

Merging not based on package path #601

Closed Floppy012 closed 1 year ago

Floppy012 commented 1 year ago

Description

When using multiple clover files regarding the same project but with different base paths, files will no longer be considered same.

Example

We split our code-testing into its different testing types (unit, component (virtual DOM), component (cypress)). These jobs produce three clover.xml files. Due to the complexity of Cypress it needs to use a container which has a different working directory path. ReportGenerator only seems to look at the file paths and not the package when merging.

In the clover files below the file app.vue is the same file (FQCN is client.app.vue for both clover files). ReportGenerator doesn't seem to use that. It uses the file path which ends up putting the file in twice in the final output.

File A (without container):

<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1680598437864" clover="3.2.0">
  <project timestamp="1680598437865" name="All files">
    <metrics statements="4748" coveredstatements="380" conditionals="2986" coveredconditionals="234" methods="1398" coveredmethods="105" elements="9132" coveredelements="719" complexity="0" loc="4748" ncloc="4748" packages="66" files="212" classes="212"/>
    <package name="client">
      <metrics statements="41" coveredstatements="0" conditionals="4" coveredconditionals="0" methods="11" coveredmethods="0"/>
      <file name="app.vue" path="/home/runner/work/redacted/redacted/client/app.vue">
        <metrics statements="19" coveredstatements="0" conditionals="2" coveredconditionals="0" methods="5" coveredmethods="0"/>
        <line num="3" count="0" type="stmt"/>
        <line num="6" count="0" type="stmt"/>
        <line num="27" count="0" type="stmt"/>
        <line num="29" count="0" type="stmt"/>
        <line num="30" count="0" type="stmt"/>
        <line num="31" count="0" type="stmt"/>
        <line num="32" count="0" type="stmt"/>
        <line num="34" count="0" type="stmt"/>
        <line num="36" count="0" type="stmt"/>
        <line num="38" count="0" type="stmt"/>
        <line num="39" count="0" type="stmt"/>
        <line num="41" count="0" type="stmt"/>
        <line num="42" count="0" type="stmt"/>
        <line num="43" count="0" type="stmt"/>
        <line num="44" count="0" type="stmt"/>
        <line num="48" count="0" type="stmt"/>
        <line num="49" count="0" type="stmt"/>
        <line num="51" count="0" type="cond" truecount="0" falsecount="2"/>
        <line num="52" count="0" type="stmt"/>
      </file>
<!-- ... -->

File B (with container):

<?xml version="1.0" encoding="UTF-8"?>
<coverage generated="1680598490506" clover="3.2.0">
  <project timestamp="1680598490507" name="All files">
    <metrics statements="85" coveredstatements="77" conditionals="34" coveredconditionals="30" methods="27" coveredmethods="22" elements="146" coveredelements="129" complexity="0" loc="85" ncloc="85" packages="77" files="254" classes="254"/>
    <package name="client">
      <metrics statements="0" coveredstatements="0" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/>
      <file name="app.vue" path="/__w/redacted/redacted/client/app.vue">
        <metrics statements="0" coveredstatements="0" conditionals="0" coveredconditionals="0" methods="0" coveredmethods="0"/>
      </file>
<!-- ... -->

Clover output of ReportGenerator

<?xml version="1.0" encoding="utf-8"?>
<coverage generated="1680598555" clover="4.3.1">
  <project timestamp="1680598555">
    <metrics complexity="0" elements="3762" coveredelements="366" conditionals="2142" coveredconditionals="1146" statements="3762" coveredstatements="366" coveredmethods="0" methods="0" classes="466" loc="22832" ncloc="3762" files="466" packages="77" />
    <package name="client">
      <metrics complexity="0" elements="39" coveredelements="0" conditionals="4" coveredconditionals="2" statements="39" coveredstatements="0" coveredmethods="0" methods="0" classes="4" loc="147" ncloc="39" files="4" />
      <file path="/__w/redacted/redacted/client/app.vue" name="app.vue">
        <metrics complexity="0" elements="0" coveredelements="0" conditionals="0" coveredconditionals="0" statements="0" coveredstatements="0" coveredmethods="0" methods="0" classes="1" loc="0" ncloc="0" />
        <class name="app.vue">
          <metrics complexity="0" elements="0" coveredelements="0" conditionals="0" coveredconditionals="0" statements="0" coveredstatements="0" coveredmethods="0" methods="0" />
        </class>
      </file>
      <file path="/home/runner/work/redacted/redacted/client/app.vue" name="app.vue">
        <metrics complexity="0" elements="18" coveredelements="0" conditionals="2" coveredconditionals="1" statements="18" coveredstatements="0" coveredmethods="0" methods="0" classes="1" loc="74" ncloc="18" />
        <class name="app.vue">
          <metrics complexity="0" elements="18" coveredelements="0" conditionals="2" coveredconditionals="1" statements="18" coveredstatements="0" coveredmethods="0" methods="0" />
        </class>
        <line num="3" count="0" type="stmt" />
        <line num="6" count="0" type="stmt" />
        <line num="27" count="0" type="stmt" />
        <line num="29" count="0" type="stmt" />
        <line num="30" count="0" type="stmt" />
        <line num="31" count="0" type="stmt" />
        <line num="32" count="0" type="stmt" />
        <line num="34" count="0" type="stmt" />
        <line num="36" count="0" type="stmt" />
        <line num="38" count="0" type="stmt" />
        <line num="39" count="0" type="stmt" />
        <line num="41" count="0" type="stmt" />
        <line num="42" count="0" type="stmt" />
        <line num="43" count="0" type="stmt" />
        <line num="44" count="0" type="stmt" />
        <line num="48" count="0" type="stmt" />
        <line num="49" count="0" type="stmt" />
        <line num="52" count="0" type="stmt" />
      </file>
<!-- ... -->
danielpalme commented 1 year ago

ReportGenerator only uses the given coverage files to determine its results. It does not analyze any further information from the file/class itself.

Reason: ReportGenerator is agnostic to the used programming language. It supports all languages (if a supported coverage file format is available). Therefore it is not aware of packages or namespaces within the code files.

In your case you can use a little Powershell script to normalize the path information. This might look like the following example:

(gc "FileB.xml") | % { $_ -replace "/__w/redacted/redacted/", "/home/runner/work/redacted/redacted/" } | Out-File "FileB.xml" -Encoding UTF8
Floppy012 commented 1 year ago

Thanks. I've already implemented the path normalization.

Correct me if I'm wrong. The <package> tag and name attribute of the <file> tag aren't language specific but rather part of the Clover Spec. That's why I thought it could be used to identify a file rather than using the path. As the path always seem to be absolute while the package hierarchy + filename always seems to be relative to the configured root directory of the reporting utility.

danielpalme commented 1 year ago

I just took a closer look.

One problem is, that some other tools also generate Clover formatted coverage files. But e.g. Jest does not comply with the "Clover Spec". It does not use <package> elements. See: https://github.com/danielpalme/ReportGenerator/pull/276

I think I will leave the current implementation as it is.