hazendaz / coveralls-maven-plugin

Maven plugin for submitting Java code coverage reports to Coveralls web service.
MIT License
3 stars 1 forks source link

coverage from integration tests not getting included in coveralls #110

Open SingingBush opened 2 months ago

SingingBush commented 2 months ago

most of the tests in my project are integrations tests run via failsafe. I've configured Jacoco to output reports for both unit tests and integration tests.

When coveralls runs I can see that it's supposedly processing both reports:

[INFO] Writing Coveralls data to /home/runner/work/myProject/myProject/target/coveralls.json...
[INFO] Processing coverage report from /home/runner/work/myProject/myProject/target/site/jacoco/jacoco.xml
[INFO] Processing coverage report from /home/runner/work/myProject/myProject/target/site/jacoco-it/jacoco.xml

but the resulting coverage in coveralls.io only gets updated with the total based on unit tests (jacoco/jacoco.xml), which is pretty low on this project.

As a workaround I've configured Jacoco to merge them both into a single report and tried using that with the plugin:

<plugin>
    <groupId>com.github.hazendaz.maven</groupId>
    <artifactId>coveralls-maven-plugin</artifactId>
    <version>4.5.0-M4</version>
    <configuration>
        <jacocoReports>
            <param>${project.reporting.outputDirectory}/jacoco-merged-report/jacoco.xml</param>
        </jacocoReports>
        <repoToken>${env.COVERALLS_REPO_TOKEN}</repoToken>
    </configuration>
</plugin>

but now in the build log I see that the plugin is actually ignoring my configuration and processing all three:

[INFO] Writing Coveralls data to /home/runner/work/myProject/myProject/target/coveralls.json...
[INFO] Processing coverage report from /home/runner/work/myProject/myProject/target/site/jacoco-merged-report/jacoco.xml
[INFO] Processing coverage report from /home/runner/work/myProject/myProject/target/site/jacoco/jacoco.xml
[INFO] Processing coverage report from /home/runner/work/myProject/myProject/target/site/jacoco-it/jacoco.xml

and the result sent to coveralls is still just the total for unit tests

SingingBush commented 2 months ago

I've also tried using the following in the config:

    <jacocoReports>
        <jacocoReport>${project.reporting.outputDirectory}/jacoco-merged-report/jacoco.xml</jacocoReport>
    </jacocoReports>
    <jacocoReports>
        <file>${project.reporting.outputDirectory}/jacoco-merged-report/jacoco.xml</file>
    </jacocoReports>
    <jacocoReports>${project.reporting.outputDirectory}/jacoco-merged-report/jacoco.xml</jacocoReports>
hazendaz commented 1 month ago

The plugin javadocs state

/**
 * File paths to additional JaCoCo coverage report files.
 */

So that is adding to what the normal reports are. Thus why all three get used.

Look here https://github.com/hazendaz/coveralls-maven-plugin/blob/e9ccb3e657ed07f7310088cb672dfa84d3760a37/src/main/java/org/eluder/coveralls/maven/plugin/util/CoverageParsersFactory.java#L96.

Basically its stating to use the additional you provided but then add the ones created from it. I think to make it just use one a different configuration option would need added. However, not sure that is necessary yet. I've never had very much luck merging files but always tried to do across platforms (ie where ran some tests on windows, others on linux). When using same system as I presume you are doing, that should work to merge them but then you would only want that one processed.

Do you have any debug information from what coveralls actually received on their end. From there it looks like we sent it. Its possible something changed. I took this plugin over as it didn't support jakarta (actually used javax without needing it). I've not spent much more time trying to fully understand the underlying code but original author did point to this and coveralls has it documented now. So this is it, so maybe their code is different. I will try my best to look into this but if you could also find out any info from their end on your scans as to what exactly happened. If we do need to just change to stop trying to add extra files and just use the one you tell it to, that should be a fairly easy change. Maybe if you want you could give it a try making such support if that seems the only logical thing to do.