Instrumentation for multi module projects

[shevek]

If I have a multi-module project where project B depends on A. I do not test A individually. I test B thoroughly; this tests A by side-effect as a JAR on the classpath. I get no reported coverage for A.

Is this possible to fix by extension somehow? (I know why it doesn't work now, but I'd like it to work!)

Thank you.

S.

[stevesaliman]

Is B a parent of A? If it is, it might be possible to simply apply the cobertura plugin to both projects A and B, which would define the instrument tasks for both projects. Then, as long as the tests for B are using the instrumented classes from A, I think the proper reports would be generated for both...

[shevek]

No, they're neighbour modules.

A good example is https://github.com/shevek/tftp4j which only really tests in tftp-server-netty and tftp-server-mina but all the engine code is common, so gets no (reported) coverage even though it's covered quite thoroughly. I particularly want coverage for the transfer state machines.

[shevek]

I mean tftp-protocol gets no apparent coverage, even though both server modules exercise the protocol and engine thoroughly.

[stevesaliman]

Pull request #65 may have fixed your issue. I tried running cobertura against your project, and I now see some coverage for classes in org.anarres.tftp.protocol classes. Can you confirm this with the latest snapshot release?

[shevek]

Looking at the PR, I don't see why it should, but I will try today. I think the classes in "other" aren't instrumented because it's hard for this plugin to tell the difference between a locally-generated JAR from another module and a generic dependency JAR downloaded from Maven Central. I confess I'm not sure how to fix this myself; I just know that the value of fixing it will be very high.

Will try later today and hope. Thank you for the note.

[shevek]

BTW, are you still publishing via Sonatype, or are you doing the Bintray thing? I tried Bintray and found it incredibly confusing and frustrating. If it's possible to set the relevant attribute via Sonatype, I'd rather go that route myself.

[stevesaliman]

I was hoping that it was a classpath issue. The PR changes the way the classpath was built, so I was hoping :-)

I had to do the Bintray thing to get the plugin into the official Gradle Plugin repository, and you're right - it is pretty confusing. It took me a few attempts to get it right, but I think I've got it now. I'm continuing to upload artifacts to Sonatype as well when I have a release.

[shevek]

I'm sorry, it didn't work. I don't see how instrumented versions of classes from sibling modules can end up on the classpath. If they do, then this will work. But the trick is, when walking the runtime dependency set, to know which JARs are from sibling modules, and which are from Maven.

[shevek]

Is there any way we might be able to move forward with this issue? It's holding us back in situations where we have code-coverage objective metrics over multi-module projects. We just did a module split, and the "apparent" (reported) coverage dropped by 5% despite the testing not actually changing.

[stevesaliman]

It's been a while since I've looked at this, so I went back and looked at it again with fresh eyes.

I think the reason you don't get any coverage of tftp-protocol is because its instrumented classes are not on the classpath for the other two projects. I think compile project(':tftp-protocol') in your build.gradle puts the contents of the protocol's sourcesets.main.output on the classpath, but not the instrumented classes in build/instrumented_classes

To get this to work properly, you'll need to have the protocol module's instrumented classes in the classpath when you run tests for the other modules, however you need to do this in a way that the instrumented classes don't wind up in your final build artifacts.

[shevek]

What I'm hoping, and I know this is a big ask, is that cobertura can somehow detect whether a given test-classpath artifact is from a sibling module, or external, and instrument the sibling JARs at the same time it instruments the classes of the current module. I know that's an order of magnitude more complex than the current logic, and I mean to battle my way through it at some point.

[benreic]

I believe I'm running into a situation described in this thread. It feels like this can be solved given the existing configuration options, but I don't think I'm setting the correct values.

Here is my projects source structure:

├── project/
│   ├── build.gradle (<- just includes ./app/build.gradle)
│   ├── app/
│   │   ├── build.gradle (<- contains the build logic for all modules)
│   ├── moduleFoo/
│   │   ├── src/
│   ├── moduleBar/
│   │   ├── src/

We have a project that contains a dozen or so submodules, with an "app" module that is a sibling project. The "app" module is what kicks off the entire build process and so all the Cobertura settings exist in ./app/build.gradle. The "app" module contains zero tests, all the tests are in the submodules (moduleFoo, moduleBar, etc.). When I run the coberturaReport task (along with my compile and test tasks), I get no cobertura outputs. The console output for the task is:

:app:coberturaReport UP-TO-DATE
:app:coberturaReport spend 0ms

I can't tell which configuration settings I need to be specifying, it seems that the relevant settings are:

• auxiliaryClasspath
• coverageDirs
• coverageSourceDirs

I've tried a variety of settings/combinations in these three, but I can't get any cobertura output to get generated.

Is there a combination of settings to include sibling code/class files into the coverage report? This does not need to be dynamically generated, I'm happy to statically include each individual module, as we won't be adding many.

Thanks for your help!