search

scoverage / scalac-scoverage-plugin

Scoverage Scala Code Coverage Core Libs
https://github.com/scoverage
Apache License 2.0
423 stars 126 forks source link

Any plan to support incremental compilation? #492

Open crakjie opened 1 year ago

crakjie commented 1 year ago

Describe the bug

Currently every time a compilation is done the datadir is cleaned. Then the scoverage.coverage is regenerated only with the compiled satements. So when the report is then generated it contains only the coverage for few statements.

It happens after incremental compilation, it happen when pulling sbt remote cache, it also happen when source files are regenerated for every build (ie file generated by https://github.com/sbt/sbt-buildinfo).

I'm not saying that having the coverage for only what was recompiled is useless. A merge request could show the percentage of coverage of it's added lines. But for project using generated source it's make it impossible to extract coverage as that compilation of generated file are done sperately.

Expected behavior

Re-calculate the scoverage.coverage for the whole source code instead, or at least having the option to do so. I think the already compiled code would not have to be re-instrumented because it was already done in the previous compilations.

What build tool are you using?

sbt

Version of scoverage

1.4.11

ckipp01 commented 1 year ago

Thanks for the report @crakjie. Could you explain a bit more of your workflow? Typically scoverage shouldn't be enabled while developing where continual incremental compilations is happening. Rather the flow is typically that coverage is enabled, which will force a full recompile and then the tests run, a report is generated, and coverage is disabled. Could you outline how you're using scoverage?

crakjie commented 1 year ago

The idea is to benefit from incremental compilation even in the CI process. Ou compilation take around 10 minutes on our small CI runner. So we push the build result with pushRemoteCache into a shared directory which is then pulled for each CI run. Because most MR impact few files most part is already compiled and compilation then take around 10s on average. Then we usually trigger another a recompilation without coverage to have a build without instrumentation (I'm not sure if it's useful).

But, as you guess, scoverage is cleaning the datadir which then make the coverage incomplete.

I also have problem with the buildInfo that are triggering a re-compilation every time because a timestamp is written in the file (but that should be managable)

lolgab commented 8 months ago

@crakjie I'm not sure if you have the same problem as I do, but I found this workaround that works well for my team: https://github.com/scoverage/sbt-scoverage/issues/309#issuecomment-1783227685