Cannot find cobertura.xml generated by sbt-scoverage

[rjenkinsjr]

We have a Scala app using sbt-scoverage to calculate code coverage and write a Cobertura-compatible XML report file.

Assume the following script is being executed in the directory /app, with the build script located at /app/build.sbt and the Scala source code package hierarchy starting at /app/src/main/scala:

#!/bin/bash
CC_TEST_REPORTER_ID=foobar
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > cc-test-reporter
chmod +x cc-test-reporter
./cc-test-reporter before-build
sbt clean coverage test coverageReport
./cc-test-reporter after-build -t cobertura -r $CC_TEST_REPORTER_ID -d --exit-code $?

1. cc-test-reporter does not find /app/target/scala-2.11/coverage-report/cobertura.xml.
2. If you move cobertura.xml into /app, cc-test-reporter fails to find the source code at /app/src/main/scala because it does not respect the <sources> element in the XML that declares where the source code was located when the report was generated.

The following script works around this issue:

#!/bin/bash
export CC_TEST_REPORTER_ID=foobar
curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > cc-test-reporter
chmod +x cc-test-reporter
./cc-test-reporter before-build
sbt clean coverage test coverageReport
cd src/main/scala
../../../cc-test-reporter format-coverage -t cobertura -d ../../../target/scala-2.11/coverage-report/cobertura.xml
../../../cc-test-reporter upload-coverage

[pbrisbin]

Hi @rjenkinsjr I believe this would be solvable by "dropping down" to lower-level format-coverage and upload-coverage commands. These commands take options to specify a) where to find the coverage file and b) what prefix any source entries will have within the coverage file, that should be stripped to make them relative to project root. I think these two things would allow the reporter to function correctly in your source tree.

I will try to spend some more time with this and come up with concrete invocations you can try, but I thought I'd mention this first in case you want to experiment yourself.

• https://github.com/codeclimate/test-reporter/blob/master/man/cc-test-reporter-format-coverage.1.md
• https://github.com/codeclimate/test-reporter/blob/master/man/cc-test-reporter-upload-coverage.1.md

[ale7714]

@rjenkinsjr hi, sorry for the late response. Thanks for sharing your workaround. I wanted to checkin with you about:

If you move cobertura.xml into /app, cc-test-reporter fails to find the source code at /app/src/main/scala because it does not respect the element in the XML that declares where the source code was located when the report was generated.

did the test reporter exited with an error? can you share it with us?

[rjenkinsjr]

@ale7714 I believe it did exit with an error, but since we've implemented that workaround on all our projects I no longer have an example on hand. I'll try to recreate the error in the next few days and post the results here for you.

[jsippel]

When we ran into this problem in circleci, the error we saw was

#!/bin/bash -eo pipefail
./cc-test-reporter after-build -t cobertura
ERRO[0000] could not find coverage file 
could not find any files in search paths for cobertura. search paths were: , cobertura.xml, cobertura.ser 
Error: could not find any files in search paths for cobertura. search paths were: , cobertura.xml, cobertura.ser
Usage:
  cc-test-reporter after-build [flags]

Flags:
  -s, --batch-size int               batch size for source files (default 500)
  -e, --coverage-endpoint string     endpoint to upload coverage information to (default "https://api.codeclimate.com/v1/test_reports")
  -t, --coverage-input-type string   type of input source to use [simplecov, lcov, coverage.py, clover, gocov, gcov, cobertura, jacoco]
      --exit-code int                exit code of the test run
  -r, --id string                    reporter identifier (default "28cbf4d6db2e993746bb3b6fff9df71d001b99e4723ea93c8883276bd6ac9d02")
  -p, --prefix string                the root directory where the coverage analysis was performed (default "/home/circleci/project")

Global Flags:
  -d, --debug   run in debug mode

Exited with code 255

[rjenkinsjr]

@ale7714 when we first started working on this, we encountered the error that @jsippel posted above.

We then tried copying cobertura.xml into the directory where we were running cc-test-reporter, which caused errors like this:

DEBU[0000] about to run format-coverage                 
DEBU[0000] using formatter cobertura                    
DEBU[0000] checking search path  for cobertura formatter 
DEBU[0000] checking search path cobertura.xml for cobertura formatter 
DEBU[0000] couldn't load branch from ENV, trying git... 
DEBU[0000] couldn't load commit sha from ENV, trying git... 
DEBU[0000] couldn't load committed at from ENV, trying git... 
DEBU[0000] getting fallback blob_id for source file path/to/SourceFile.scala 
ERRO[0000] failed to read file path/to/SourceFile.scala
open path/to/SourceFile.scala: no such file or directory 
Error: open path/to/SourceFile.scala: no such file or directory
Usage:
  cc-test-reporter after-build [flags]

Flags:
  -s, --batch-size int               batch size for source files (default 500)
  -e, --coverage-endpoint string     endpoint to upload coverage information to (default "https://api.codeclimate.com/v1/test_reports")
  -t, --coverage-input-type string   type of input source to use [simplecov, lcov, coverage.py, clover, gocov, gcov, cobertura, jacoco]
      --exit-code int                exit code of the test run
  -r, --id string                    reporter identifier
  -p, --prefix string                the root directory where the coverage analysis was performed (default "/app")

Global Flags:
  -d, --debug   run in debug mode

Next, we tried editing cobertura.xml to manually update the filenames written there (obviously not intended as a permanent fix). For example, we would change elements like this...

<class 
name="path.to.SourceFile" filename="path/to/SourceFile.scala" line-rate="0.10" branch-rate="1.00" complexity="0">

...to this...

<class 
name="path.to.SourceFile" filename="src/main/scala/path/to/SourceFile.scala" line-rate="0.10" branch-rate="1.00" complexity="0">

This fixed the issue, and led us to the workaround I posted.

This is our cobertura.xml, with package/class names changed. The <sources> element provides the absolute path prefix from which all <class filename=""> values can be resolved relatively. (Our build was run in /app.) Looks like cc-test-reporter needs to take <sources> into account when it exists.

[ale7714]

@rjenkinsjr Thank you for sharing your cobertura file. Sorry i didn't have an update sooner. We're tracking this and will update our Cobertura formatter to take into account the tag. I'll let you know when an update with this fixed is released.

@jsippel thank you for writing. Currently cc-test-reporter will try to share the Cobertura file within your project folder but many tools store the coverage data outside the project folder. For those cases you can use the low level commands to indicate where the coverage data is and upload to CC:

./cc-test-reporter format-coverage -t cobertura path/to/coverage_data
./cc-test-reporter upload-coverage

We're working on improving our docs and adding example for projects using sbt-scoverage. Let us know if you have more questions.

[netflash]

Hi there,

I have the similar problem, but with jacoco formatter. my workaround is almost the same as the one mentioned in the first message

cd src/main/java/com/orgname/appname
../../../../../../cc-test-reporter format-coverage -t jacoco -d ../../../../../../build/reports/jacoco/test/jacocoTestReport.xml

it's better that nothing, but still doesn't work as the filename.java could be in a sub-directory

and BTW the reporter searches for jacoco.xml, not the jacocoTestReport.xml.

I won't create a separate issue for this - as I'm just evaluating codeclimate for now.

[ale7714]

@netflash 👋 thanks for reporting this. Can you share an example of your coverage file? it would be helpful. Currently the test-reporter is able to infer a handful of default coverage filenames but it's not an exhaustive list yet. That's why we offer the option to pass the path to the coverage file as an argument to the test reporter. Also, addingjacocoTestReport.xml to the current searchPaths is fairly easy. PRs are welcome :).

[ale7714]

@rjenkinsjr from v0.2.1 and up test-reporter should take into account the source tags from the cobertura file. Also you should be able to remove the cd command your CI script and run

./cc-test-reporter before-build
sbt clean coverage test coverageReport
./cc-test-reporter format-coverage -t cobertura -d target/scala-2.11/coverage-report/cobertura.xml
./cc-test-reporter upload-coverage

Let me know if you have any other question.

[ale7714]

Closing this as the issue was addressed. Feel free to open a new issue if anything comes up.