search

bazelbuild / rules_scala

Scala rules for Bazel
Apache License 2.0
364 stars 275 forks source link

code coverage support #184

Open johnynek opened 7 years ago

johnynek commented 7 years ago

This would look like using:

https://github.com/scoverage/scalac-scoverage-plugin

to get the scala test rules to emit code coverage reports, then aggregating them across the repo.

It is related to https://github.com/bazelbuild/bazel/issues/1118 but I don't think it is actually a blocker. We may just be able to produce coverage output of each test rule, then an aspect to walk all the rules and aggregate a total.

gergelyfabian commented 4 years ago

I believe this issue could be closed as Jacoco code coverage for Scala works.

ittaiz commented 4 years ago

Thanks! from your previous post it sounds like it works but in awkward way. Is this bazel's fault, rules_scala's fault or just how coverage business works? (We don't do coverage internally so I don't really know)

softprops commented 4 years ago

I believe this issue could be closed as Jacoco code coverage for Scala works.

If we have a working example can we document it. Some times as passed so things may have changed but this is where we're currently blocked

scala_rules@5261499b0485f33799a1b210796fcdfa720a5344 bazel@1.2.1

bazel coverage //...
[snip]
*** RUN ABORTED *** (857 milliseconds)
  java.lang.NoClassDefFoundError: org/jacoco/agent/rt/internal_1f1cc91/Offline
  at com.meetup.base.secret.Secrets.<clinit>(Secrets.java)
  at sun.reflect.GeneratedSerializationConstructorAccessor1.newInstance(Unknown Source)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:45)
  at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:73)
  at org.mockito.internal.creation.instance.ObjenesisInstantiator.newInstance(ObjenesisInstantiator.java:14)
  at org.mockito.internal.creation.cglib.ClassImposterizer.createProxy(ClassImposterizer.java:143)
  at org.mockito.internal.creation.cglib.ClassImposterizer.imposterise(ClassImposterizer.java:58)
  at org.mockito.internal.creation.cglib.ClassImposterizer.imposterise(ClassImposterizer.java:49)
  at org.mockito.internal.creation.cglib.CglibMockMaker.createMock(CglibMockMaker.java:24)
  at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:33)
  at org.mockito.internal.MockitoCore.mock(MockitoCore.java:59)
  at org.mockito.Mockito.mock(Mockito.java:1285)
  at org.mockito.Mockito.mock(Mockito.java:1163)
  at org.scalatest.mockito.MockitoSugar$class.mock(MockitoSugar.scala:73)
  at com.meetup.base.db.pool.ConnectionPoolTest.mock(ConnectionPoolTest.scala:14)
  at com.meetup.base.db.pool.ConnectionPoolTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$3.apply(ConnectionPoolTest.scala:18)
  at com.meetup.base.db.pool.ConnectionPoolTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$3.apply(ConnectionPoolTest.scala:17)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  at org.scalatest.FunSpecLike$$anon$1.apply(FunSpecLike.scala:454)
  at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
  at org.scalatest.FunSpec.withFixture(FunSpec.scala:1630)
  at org.scalatest.FunSpecLike$class.invokeWithFixture$1(FunSpecLike.scala:451)
  at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
  at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
  at org.scalatest.FunSpecLike$class.runTest(FunSpecLike.scala:464)
  at org.scalatest.FunSpec.runTest(FunSpec.scala:1630)
  at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
  at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:373)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:410)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:373)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:410)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
  at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
  at org.scalatest.FunSpecLike$class.runTests(FunSpecLike.scala:497)
  at org.scalatest.FunSpec.runTests(FunSpec.scala:1630)
  at org.scalatest.Suite$class.run(Suite.scala:1147)
  at org.scalatest.FunSpec.org$scalatest$FunSpecLike$$super$run(FunSpec.scala:1630)
  at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
  at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
  at org.scalatest.SuperEngine.runImpl(Engine.scala:521)
  at org.scalatest.FunSpecLike$class.run(FunSpecLike.scala:501)
  at org.scalatest.FunSpec.run(FunSpec.scala:1630)
  at org.scalatest.Suite$class.callExecuteOnSuite$1(Suite.scala:1210)
  at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1257)
  at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1255)
  at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
  at org.scalatest.Suite$class.runNestedSuites(Suite.scala:1255)
  at org.scalatest.tools.DiscoverySuite.runNestedSuites(DiscoverySuite.scala:30)
  at org.scalatest.Suite$class.run(Suite.scala:1144)
  at org.scalatest.tools.DiscoverySuite.run(DiscoverySuite.scala:30)
  at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
  at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1346)
  at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1340)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
  at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1011)
  at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1010)
  at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
  at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
  at org.scalatest.tools.Runner$.main(Runner.scala:827)
  at org.scalatest.tools.Runner.main(Runner.scala)
  at io.bazel.rulesscala.scala_test.Runner.main(Runner.java:34)
  Cause: java.lang.ClassNotFoundException: org.jacoco.agent.rt.internal_1f1cc91.Offline
  at java.net.URLClassLoader.findClass(URLClassLoader.java:382)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:418)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:351)
  at com.meetup.base.secret.Secrets.<clinit>(Secrets.java)
  at sun.reflect.GeneratedSerializationConstructorAccessor1.newInstance(Unknown Source)
  at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
  at org.objenesis.instantiator.sun.SunReflectionFactoryInstantiator.newInstance(SunReflectionFactoryInstantiator.java:45)
  at org.objenesis.ObjenesisBase.newInstance(ObjenesisBase.java:73)
  at org.mockito.internal.creation.instance.ObjenesisInstantiator.newInstance(ObjenesisInstantiator.java:14)
  at org.mockito.internal.creation.cglib.ClassImposterizer.createProxy(ClassImposterizer.java:143)
  at org.mockito.internal.creation.cglib.ClassImposterizer.imposterise(ClassImposterizer.java:58)
  at org.mockito.internal.creation.cglib.ClassImposterizer.imposterise(ClassImposterizer.java:49)
  at org.mockito.internal.creation.cglib.CglibMockMaker.createMock(CglibMockMaker.java:24)
  at org.mockito.internal.util.MockUtil.createMock(MockUtil.java:33)
  at org.mockito.internal.MockitoCore.mock(MockitoCore.java:59)
  at org.mockito.Mockito.mock(Mockito.java:1285)
  at org.mockito.Mockito.mock(Mockito.java:1163)
  at org.scalatest.mockito.MockitoSugar$class.mock(MockitoSugar.scala:73)
  at com.meetup.base.db.pool.ConnectionPoolTest.mock(ConnectionPoolTest.scala:14)
  at com.meetup.base.db.pool.ConnectionPoolTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$3.apply(ConnectionPoolTest.scala:18)
  at com.meetup.base.db.pool.ConnectionPoolTest$$anonfun$1$$anonfun$apply$mcV$sp$1$$anonfun$apply$mcV$sp$3.apply(ConnectionPoolTest.scala:17)
  at org.scalatest.OutcomeOf$class.outcomeOf(OutcomeOf.scala:85)
  at org.scalatest.OutcomeOf$.outcomeOf(OutcomeOf.scala:104)
  at org.scalatest.Transformer.apply(Transformer.scala:22)
  at org.scalatest.Transformer.apply(Transformer.scala:20)
  at org.scalatest.FunSpecLike$$anon$1.apply(FunSpecLike.scala:454)
  at org.scalatest.TestSuite$class.withFixture(TestSuite.scala:196)
  at org.scalatest.FunSpec.withFixture(FunSpec.scala:1630)
  at org.scalatest.FunSpecLike$class.invokeWithFixture$1(FunSpecLike.scala:451)
  at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
  at org.scalatest.FunSpecLike$$anonfun$runTest$1.apply(FunSpecLike.scala:464)
  at org.scalatest.SuperEngine.runTestImpl(Engine.scala:289)
  at org.scalatest.FunSpecLike$class.runTest(FunSpecLike.scala:464)
  at org.scalatest.FunSpec.runTest(FunSpec.scala:1630)
  at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
  at org.scalatest.FunSpecLike$$anonfun$runTests$1.apply(FunSpecLike.scala:497)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:396)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:373)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:410)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:373)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:410)
  at org.scalatest.SuperEngine$$anonfun$traverseSubNodes$1$1.apply(Engine.scala:384)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.SuperEngine.traverseSubNodes$1(Engine.scala:384)
  at org.scalatest.SuperEngine.org$scalatest$SuperEngine$$runTestsInBranch(Engine.scala:379)
  at org.scalatest.SuperEngine.runTestsImpl(Engine.scala:461)
  at org.scalatest.FunSpecLike$class.runTests(FunSpecLike.scala:497)
  at org.scalatest.FunSpec.runTests(FunSpec.scala:1630)
  at org.scalatest.Suite$class.run(Suite.scala:1147)
  at org.scalatest.FunSpec.org$scalatest$FunSpecLike$$super$run(FunSpec.scala:1630)
  at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
  at org.scalatest.FunSpecLike$$anonfun$run$1.apply(FunSpecLike.scala:501)
  at org.scalatest.SuperEngine.runImpl(Engine.scala:521)
  at org.scalatest.FunSpecLike$class.run(FunSpecLike.scala:501)
  at org.scalatest.FunSpec.run(FunSpec.scala:1630)
  at org.scalatest.Suite$class.callExecuteOnSuite$1(Suite.scala:1210)
  at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1257)
  at org.scalatest.Suite$$anonfun$runNestedSuites$1.apply(Suite.scala:1255)
  at scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:33)
  at scala.collection.mutable.ArrayOps$ofRef.foreach(ArrayOps.scala:186)
  at org.scalatest.Suite$class.runNestedSuites(Suite.scala:1255)
  at org.scalatest.tools.DiscoverySuite.runNestedSuites(DiscoverySuite.scala:30)
  at org.scalatest.Suite$class.run(Suite.scala:1144)
  at org.scalatest.tools.DiscoverySuite.run(DiscoverySuite.scala:30)
  at org.scalatest.tools.SuiteRunner.run(SuiteRunner.scala:45)
  at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1346)
  at org.scalatest.tools.Runner$$anonfun$doRunRunRunDaDoRunRun$1.apply(Runner.scala:1340)
  at scala.collection.immutable.List.foreach(List.scala:392)
  at org.scalatest.tools.Runner$.doRunRunRunDaDoRunRun(Runner.scala:1340)
  at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1011)
  at org.scalatest.tools.Runner$$anonfun$runOptionallyWithPassFailReporter$2.apply(Runner.scala:1010)
  at org.scalatest.tools.Runner$.withClassLoaderAndDispatchReporter(Runner.scala:1506)
  at org.scalatest.tools.Runner$.runOptionallyWithPassFailReporter(Runner.scala:1010)
  at org.scalatest.tools.Runner$.main(Runner.scala:827)
  at org.scalatest.tools.Runner.main(Runner.scala)
  at io.bazel.rulesscala.scala_test.Runner.main(Runner.java:34)
gergelyfabian commented 4 years ago

Thanks! from your previous post it sounds like it works but in awkward way. Is this bazel's fault, rules_scala's fault or just how coverage business works? (We don't do coverage internally so I don't really know)

  1. bazel does not summarize the coverage statistics for you - bazel's or rules_scala's fault, I cannot tell which
  2. when you try running genhtml for the .dat files it won't have access to the source files - this seems to be a general JVM issue - , please check the Gerrit ticket with Bazel I quoted: https://github.com/bazelbuild/bazel/issues/2528
gergelyfabian commented 4 years ago

I believe this issue could be closed as Jacoco code coverage for Scala works.

If we have a working example can we document it. Some times as passed so things may have changed but this is where we're currently blocked

scala_rules@5261499b0485f33799a1b210796fcdfa720a5344 bazel@1.2.1

Your rules_scala version seems to be a bit old (August 2019). I tried with 26cf9b74fc46f1e9a970c97837447549ed7257b6, that is from November 2019.

Also, I used rules_jvm_external 3.1 for this test to add the JVM dependencies.

gergelyfabian commented 4 years ago

Here is a working example:

https://github.com/gergelyfabian/bazel-scala-example

Run:

bazel coverage --extra_toolchains="@io_bazel_rules_scala//test/coverage:enable_code_coverage_aspect" //...

EDIT: example was updated with multiple targets generating code coverage and a demonstration how to use genhtml (with a script inspired by Gerrit project).

ittaiz commented 4 years ago

Maybe this could be a documentation PR?

On Thu, 2 Jan 2020 at 18:13 Gergely Fábián notifications@github.com wrote:

Here is a working example:

https://github.com/gergelyfabian/bazel-scala-example

Run:

bazel coverage --extra_toolchains="@io_bazel_rules_scala//test/coverage:enable_code_coverage_aspect" //...

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/184?email_source=notifications&email_token=AAKQQF3R7JU6FMP5NRSQWVLQ3YG4BA5CNFSM4DI3PEGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEH6WPQQ#issuecomment-570255298, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKQQF5NLTKMQC4RHIFEYPTQ3YG4BANCNFSM4DI3PEGA .

--

Ittai Zeidman

40 Hanamal street, Tel Aviv, Israel

http://www.wix.com

gergelyfabian commented 4 years ago

Maybe this could be a documentation PR?

Sure :) However, please let me know what you mean by that, as I'm a bazel beginner :) How should I create a documentation PR, and for which component? rules_scala I guess?

ittaiz commented 4 years ago

I mean send a PR to rules_scala which adds a detailed explanation on how to use coverage. It sounds like this can be lengthy so maybe have a separate markdown file and link to it from the main readme?

On Tue, 7 Jan 2020 at 13:25 Gergely Fábián notifications@github.com wrote:

Maybe this could be a documentation PR?

Sure :) However, please let me know what you mean by that, as I'm a bazel beginner :) How should I create a documentation PR, and for which component? rules_scala I guess?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/bazelbuild/rules_scala/issues/184?email_source=notifications&email_token=AAKQQF2W3IM224GLIQPMKSLQ4RRBXA5CNFSM4DI3PEGKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEIIR7JY#issuecomment-571547559, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAKQQF6GB5RMABE3E2OUMELQ4RRBXANCNFSM4DI3PEGA .

--

Ittai Zeidman

40 Hanamal street, Tel Aviv, Israel

http://www.wix.com

d-haxton commented 4 years ago

Just a quick update for anyone still paying attention: https://github.com/bazelbuild/rules_scala/pull/1006

This should completely remove the need to run any scripts to fix file paths. You should just be able to run genhtml from your directory and passing in the coverage.dat (although it will add noise to your code, but that's a different conversations :joy:)

gergelyfabian commented 4 years ago

Just a quick update for anyone still paying attention: #1006

This should completely remove the need to run any scripts to fix file paths. You should just be able to run genhtml from your directory and passing in the coverage.dat (although it will add noise to your code, but that's a different conversations )

After merging #1006 to master indeed code coverage support in rules_scala become a lot better. I confirm that for my usecase indeed simply running genhtml works without any path fixes.

ittaiz commented 4 years ago

@gergelyfabian great! Wdyt about sending a PR to the readme of what one should do to get coverage? Also this is only for scalatest Targets, right? I think junit/specs2 should be low cost but AFAIU they aren’t currently supported

gergelyfabian commented 4 years ago

Here is the PR for documentation on coverage: https://github.com/bazelbuild/rules_scala/pull/1017

I only tested with scalatest.

tanishiking commented 1 year ago

Regarding scoverage integration: FYI, as commented in

The scala side should be easy. What I don't know is how to plug in to bazel's expectations (if they are even standard). Like where should we write the coverage files to? What should the format be? https://github.com/bazelbuild/rules_scala/issues/184#issuecomment-299544682

the scala side isn't hard, we can create an extra instrumented version of build target, then it writes to the location we specified. like this https://github.com/tanishiking/bazel-playground/blob/main/18-scala-scoverage

The thing is "how to plug in to bazel's expectations (if they are even standard)", and maybe now we have good references

sluongng commented on May 11, 2021 Good references should be from rules_go: https://github.com/bazelbuild/rules_go/issues/140 (and some PRs mentioned here) https://docs.google.com/document/d/1-ZWHF-Q-qCKf19ik-t33ie58BkNurrYYzKR4OLtcilY/edit Generally Bazel's coverage is not yet complete. If your code coverage tooling does not follow the Java's convention, you gona have a bad time. https://github.com/bazelbuild/rules_rust/issues/690#issuecomment-837872013

tanishiking commented 1 year ago

Hey, I've been trying to get rules_scala to integrate well with scoverage, and I've found that it's not possible with the current scoverage implementation 😞 I'll leave some insights I gained during my struggle.

https://github.com/tanishiking/bazel-playground/tree/main/18-scala-scoverage

How scoverage-scalac-plugin works?

While JaCoCo instrument the compiled bytecode, Scoverage instrument against the source code (Scala AST) in the very early stages of the compilation phases.

To instrument the given scala code, we'll provde the following scalacopts

-Xplugin:/path/to/plugin/scalac-scoverage-plugin_1.4.11.jar # for scoverage >2, we have to add some more deps
-P:scoverage:dataDir:/path/to/project/output
-P:scoverage:sourceRoot:/path-to/project/root

Watch @ckipp01's video for more details 👍 https://www.youtube.com/watch?v=SIkNgemGmYQ

Naive integration

So, I tried to make PoC scoverage + rules_scala works here

https://github.com/tanishiking/bazel-playground/tree/main/18-scala-scoverage

Even though it doesn't follow "bazel way" (it generates coverage report by bazel test, tests have to depends on instrumented targets, and it writes things into somewhere outside output base), it somehow "works".

However, there're plenty of problems with this implementation, of course.

(1) scoverage.coverage won't be cached.

Since we specify -P:scoverage:dataDir:/tmp/... and we don't register them as outputs, they will of course not be cached. As a result, when the build is cached, scoverage.coverage won't be available and we won't be able to do any coverage aggregation. (Actually, the dataDir won't be created and invokers will fail to write measurements).

We have to make scoverage.coverage to be cached.

Suppose we're trying to add/replace a phase using customized_phase.

That being said, it seems like we may want to scoverage-scalac-plugin to write scoverage.coverage information into output JAR file together?

(2) scoverage.measurements won't be cached.

Since scoverage.measurements also not registered as rule's output, they won't be cached too. As a result, if a test is cached, the coverage data also won't be available.

The problem is scoverage-scalac-plugin will write measurements files into dataDir specified in target under test, tests can't know where the measurements files will be written.

(3) How can we make it work with bazel coverage ???

How/when should we instrument the code?

For JaCoCo, Bazel will instrument the compiled bytecode and create .uninstrumented class file, and normal class files.

https://docs.google.com/document/d/1mQLQa2uMIgVcwTE-hwaKgwQ5upExIRH5p9lHFyxhj4g/edit#heading=h.sfpf1dphkfqb

However, tools that instrument against source code such as scoverage and cargo-tarpaulin for Rust making two (instrumented and uninstrumented) compiled artifact means we have to run compilation twice, which seems not acceptable, especially for Scala (which isn't compile that fast).

alexmtrmd commented 1 year ago

+1