Branch coverage gets ignored (Request: Support for Branch Coverage)

[draialexis]

In a Java 21 / Spring Boot project using Gradle with JUnit and JaCoCo, I've noticed something.

New unit tests can get thrown away because "coverage did not increase", when in fact the test would increase branch coverage -- if not line coverage.

Here's an example: (please do ignore the wonky casing and the dummy test that says 'assertTrue(true)')

In that example, the case where the collection is null is a different case than when the collection is empty. I believe the extra test is indeed valuable, and should be kept -- since it increases the branch coverage metric.

Besides, if a test were to increase complexity coverage, or method coverage, I would rather Cover Agent not throw it away either: users can then decide which tests they want to keep.

Looking at a JaCoCo report (in .csv here), we can see that those metrics are indeed available:

...CLASS,INSTRUCTION_MISSED,INSTRUCTION_COVERED,BRANCH_MISSED,BRANCH_COVERED,LINE_MISSED,LINE_COVERED,COMPLEXITY_MISSED,COMPLEXITY_COVERED,METHOD_MISSED,METHOD_COVERED
...AbstractService,131,69,11,3,44,25,25,9,18,9
...CrudDatableService,169,0,8,0,49,0,27,0,23,0
...AbstractEntity,13,25,3,3,1,7,4,2,1,2
...

I didn't see any issue templates here, so I wrote something basic. Please feel free to let me know if you do want me to adhere to specific guidelines, and I'll do what I can. Please let me know if you want any extra information. Thanks for putting this out there, I've envoyed testing it out so far

[EmbeddedDevops1]

@draialexis This is awesome. Thanks so much for opening up this issue. Branch coverage (and a myriad of other types of coverage) is something we absolutely want in Cover Agent.

We'd need to support it for the existing supported coverage report types (right now just Cobertura and JaCoCo) but it's definitely something we'd like to add.

If you'd like to take a stab at it you could create the parsing functions in cover_agent/CoverageProcessor.py and see how it fairs. After that I could integrate it into cover_agent/UnitTestGenerator.py or you could take try as well.

Regarding the template: This was great. Thanks again for contributing to this community's project!

[draialexis]

Thanks for the response!

I'm not super proficient with Python and I don't have a lot of free time these days, so I don't think I'll be taking a stab at it myself.

Initially I was thinking about trying, because I thought I would be able to simply substitute line coverage for branch coverage, rename some stuff accordingly, and send my PR.

That's because branch coverage subsumes statement coverage (cf. ISTQB Certified Tester - Foundation Level Syllabus v4.0, Section 4.3.2. Branch Testing and Branch Coverage)

But looking more closely at examples from different coverage reports like JaCoCo and cobertura, it's possible to see 100% branch coverage and <100% line / statement / instruction coverage metrics. This indicates to me that they use specific approaches in the ways they calculate those metrics, making it so branch coverage no longer subsumes statement coverage.

If I'm correct, then that means the task at hand is no longer to substitute one metric for another, but to switch from a 1-metric paradigm (line coverage) to an n>=1-metric paradigm -- I haven't read all the code, but I'll bet that it's not going to be trivial

[EmbeddedDevops1]

@draialexis Got a POC up for branch coverage. Want to give it a test? https://github.com/Codium-ai/cover-agent/pull/196

[draialexis]

@EmbeddedDevops1 thanks, I’d gladly give it a spin! I’ll try to fit that in tomorrow.

[draialexis]

Just in case someone stumbles upon this issue and misses the PR (where I gave feedback), here it is: https://github.com/Codium-ai/cover-agent/pull/196

[EmbeddedDevops1]

Just in case someone stumbles upon this issue and misses the PR (where I gave feedback), here it is: #196

Heads up: I re-opened a new PR: https://github.com/Codium-ai/cover-agent/pull/228