search

trautonen / coveralls-maven-plugin

Maven plugin for submitting Java code coverage reports to Coveralls web service.
MIT License
312 stars 123 forks source link

Wrong coverage reported with a merged jacoco.exec file #86

Open Zlika opened 8 years ago

Zlika commented 8 years ago

Hi,

The coverage reported by Coveralls is very different from the one reported by Jacoco. My project is this one: https://github.com/Zlika/reproducible-build-maven-plugin I have unit tests and integration tests. The 2 jacoco.exec files produced are merged into a single jacoco.exec file (using the "merge" goal of the jacoco-maven-plugin). From this file, I produce a Jacoco report and I also use coveralls-maven-plugin to send the coverage to coveralls. However, the coverage reported by coveralls (cf. https://coveralls.io/builds/3774697) is very different from the one on the jacoco report (cf. http://zlika.github.io/reproducible-build-maven-plugin/jacoco/io.github.zlika.reproducible/index.html). In particular, all the classes that are only tested during the integration tests shows a 0% coverage with coveralls (e.g. StripJaxbMojo.java and StripJarMojo.java). It seems like coveralls only takes into account the coverage of the unit tests, but I don't know how it's possible because the jacoco report is ok and takes into account both unit and integration tests.

Regards, Thomas

trautonen commented 8 years ago

This is very good question indeed why a build from travis results wrong coverage. I cloned the project locally and ran the same commands you have in your travis.yml and results were:

[INFO] Writing Coveralls data to /tmp/reproducible-build-maven-plugin/target/coveralls.json...
[INFO] Processing coverage report from /tmp/reproducible-build-maven-plugin/target/site/jacoco/jacoco.xml
[INFO] Successfully wrote Coveralls data in 119ms
[INFO] Gathered code coverage metrics for 9 source files with 708 lines of code:
[INFO] - 178 relevant lines
[INFO] - 162 covered lines
[INFO] - 16 missed lines

Also verified from the generated coveralls.json that for example StripJaxbMojo had correctly covered lines.

All in all. I don't know why the report differs from what I've run locally. Travis is quite hard to debug. You could try to find what's different in the Travis build logs compared to local run with -DdryRun=true option.

Zlika commented 8 years ago

Thank you for your answer. The travis log is indeed different:

[INFO] Writing Coveralls data to /home/travis/build/Zlika/reproducible-build-maven-plugin/target/coveralls.json...
[INFO] Processing coverage report from /home/travis/build/Zlika/reproducible-build-maven-plugin/target/site/jacoco/jacoco.xml
[INFO] Successfully wrote Coveralls data in 187ms
[INFO] Gathered code coverage metrics for 9 source files with 708 lines of code:
[INFO] - 178 relevant lines
[INFO] - 112 covered lines
[INFO] - 66 missed lines

I will try to analyze travis logs in depth to see what is different from a local build.

electronicpeter commented 5 years ago

How did you solve your problem. I currently have the same. Locally everything is fine and the jacoco.xml has the correct number. In travis all covered lines are zero (as I have a project that only contains the "merged" numbers.