ckipp01
commented
2 years ago Hey @longliveenduro do you still have this issue using the latest sbt-scoverage? If so could you provide a minimal project for me to reproduce this?
Open longliveenduro opened 2 years ago
ckipp01
commented
2 years ago Hey @longliveenduro do you still have this issue using the latest sbt-scoverage? If so could you provide a minimal project for me to reproduce this?
longliveenduro
commented
2 years ago Thanks @ckipp01 for coming back to this. The problem still exists in 2.0.4, see this to reproduce it: https://github.com/longliveenduro/zio-geolocation-tapir-tapir-starter
sbt clean coverage test coverageReport produces:
chris@neptun:~/prj/zio-geolocation-tapir-tapir-starter$ sbt clean coverage test coverageReport
[info] welcome to sbt 1.7.1 (Private Build Java 17.0.4)
[info] loading settings for project global-plugins from sbt-updates.sbt ...
[info] loading global plugins from /home/chris/.sbt/1.0/plugins
[info] loading settings for project zio-geolocation-tapir-tapir-starter-build from plugins.sbt ...
[info] loading project definition from /home/chris/prj/zio-geolocation-tapir-tapir-starter/project
[info] loading settings for project rootProject from build.sbt ...
[info] set current project to zio-geolocation-tapir (in build file:/home/chris/prj/zio-geolocation-tapir-tapir-starter/)
[success] Total time: 0 s, completed Sep 28, 2022, 7:00:07 PM
[info] Defining ThisBuild / coverageEnabled
[info] The new value will be used by Compile / compile / scalacOptions, libraryDependencies
[info] Reapplying settings...
[info] set current project to zio-geolocation-tapir (in build file:/home/chris/prj/zio-geolocation-tapir-tapir-starter/)
[info] compiling 2 Scala sources to /home/chris/prj/zio-geolocation-tapir-tapir-starter/target/scala-3.2.0/classes ...
[info] compiling 1 Scala source to /home/chris/prj/zio-geolocation-tapir-tapir-starter/target/scala-3.2.0/test-classes ...
+ Endpoints spec
+ return hello message
+ list available books
2 tests passed. 0 tests failed. 0 tests ignored.
Executed in 784 ms
[info] Completed tests
[success] Total time: 10 s, completed Sep 28, 2022, 7:00:18 PM
[info] Waiting for measurement data to sync...
[info] Reading scoverage instrumentation [/home/chris/prj/zio-geolocation-tapir-tapir-starter/target/scala-3.2.0/scoverage-data/scoverage.coverage]
[info] Reading scoverage measurements...
[info] Generating scoverage reports...
[error] java.lang.RuntimeException: No source root found for '/home/chris/prj/zio-geolocation-tapir-tapir-starter/src/core/impl.scala' (source roots: '/home/chris/prj/zio-geolocation-tapir-tapir-starter/src/main/scala/')
[error] at scoverage.reporter.BaseReportWriter.relativeSource(BaseReportWriter.scala:35)
[error] at scoverage.reporter.BaseReportWriter.relativeSource(BaseReportWriter.scala:23)
[error] at scoverage.reporter.CoberturaXmlWriter.klass(CoberturaXmlWriter.scala:54)
[error] at scoverage.reporter.CoberturaXmlWriter.$anonfun$pack$1(CoberturaXmlWriter.scala:78)
[error] at scala.collection.TraversableLike.$anonfun$map$1(TraversableLike.scala:286)
[error] at scala.collection.immutable.List.foreach(List.scala:431)
[error] at scala.collection.TraversableLike.map(TraversableLike.scala:286)
[error] at scala.collection.TraversableLike.map$(TraversableLike.scala:279)
[error] at scala.collection.immutable.List.map(List.scala:305)
[error] at scoverage.reporter.CoberturaXmlWriter.pack(CoberturaXmlWriter.scala:78)
[error] at scoverage.reporter.CoberturaXmlWriter.$anonfun$xml$3(CoberturaXmlWriter.scala:106)
[error] at scala.collection.immutable.List.map(List.scala:293)
[error] at scoverage.reporter.CoberturaXmlWriter.xml(CoberturaXmlWriter.scala:106)
[error] at scoverage.reporter.CoberturaXmlWriter.write(CoberturaXmlWriter.scala:30)
[error] at scoverage.ScoverageSbtPlugin$.writeReports(ScoverageSbtPlugin.scala:296)
[error] at scoverage.ScoverageSbtPlugin$.$anonfun$coverageReport0$1(ScoverageSbtPlugin.scala:228)
[error] at scoverage.ScoverageSbtPlugin$.$anonfun$coverageReport0$1$adapted(ScoverageSbtPlugin.scala:203)
[error] at scala.Function1.$anonfun$compose$1(Function1.scala:49)
[error] at sbt.internal.util.$tilde$greater.$anonfun$$u2219$1(TypeFunctions.scala:62)
[error] at sbt.std.Transform$$anon$4.work(Transform.scala:68)
[error] at sbt.Execute.$anonfun$submit$2(Execute.scala:282)
[error] at sbt.internal.util.ErrorHandling$.wideConvert(ErrorHandling.scala:23)
[error] at sbt.Execute.work(Execute.scala:291)
[error] at sbt.Execute.$anonfun$submit$1(Execute.scala:282)
[error] at sbt.ConcurrentRestrictions$$anon$4.$anonfun$submitValid$1(ConcurrentRestrictions.scala:265)
[error] at sbt.CompletionService$$anon$2.call(CompletionService.scala:64)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:539)
[error] at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
[error] at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
[error] at java.base/java.lang.Thread.run(Thread.java:833)
[error] (coverageReport) No source root found for '/home/chris/prj/zio-geolocation-tapir-tapir-starter/src/core/impl.scala' (source roots: '/home/chris/prj/zio-geolocation-tapir-tapir-starter/src/main/scala/')
[error] Total time: 1 s, completed Sep 28, 2022, 7:00:19 PMSorry it took so long to get back to this @longliveenduro . Looking closer I have a feeling that whatever is impacting https://github.com/lampepfl/dotty/issues/15791 is the same issue here. Looking through what's generated in the scoverage.coverage file I'm seeing stuff generated that shouldn't be. Here are a couple examples:
4
core/src/main/scala-3/sttp/tapir/internal/CodecValueClassMacro.scala
toil
Endpoints$
Object
toil.Endpoints$
$anonfun
877
902
26
<init>
Apply
false
0
falseThis looks to be actually instrumenting some macro code inside of tapir. And then related to the error you're seeing:
11
src/core/impl.scala
toil
Endpoints$
Object
toil.Endpoints$
<init>
435
438
19
Map
Ident
false
0
falseThis also shouldn't be implemented. Feel free to create a report for this inside of the Dotty repo as it will need to be fixed there. I'm not 100% sure it is the same as the one I listed so feel free to just create a new one and then leave it up to them to mark it as a duplicate or not.
TheElectronWill
commented
2 years ago I'm preparing a PR to dotty with coverage bugfixes, it will hopefully fix this issue :)
TheElectronWill
commented
2 years ago longliveenduro
commented
1 year ago @TheElectronWill I upgraded to Scala 3.2.1 and sbt scoverage to 2.0.5 on https://github.com/longliveenduro/zio-geolocation-tapir-tapir-starter.
And still the same error occurs
TheElectronWill
commented
1 year ago Ah! I should add something on the sbt-coverage side then. Thanks for warning me :)
longliveenduro
commented
1 year ago Thank you very much for your effort. Just realized that sbt-scoverage 2.0.6 is already out. Updated my example to reproduce the problem.
longliveenduro
commented
1 year ago FYI: in our closed source code we have now this when using sbt coverage test:
[info] compiling 41 Scala sources to /home/chris/prj/toil/zio-geolocation/modules/core/target/scala-3.2.0/classes ...
[info] compiling 37 Scala sources to /home/chris/prj/toil/zio-geolocation/modules/core/target/scala-3.2.0/test-classes ...
[error] Uncaught exception when running tests: java.lang.VerifyError: Constructor must call super() or this() before return
[error] Exception Details:
[error] Location:
[error] com/tsystems/toil/geolocation/model/RefinedTypes$UtmZone$.<init>()V @0: return
[error] Reason:
[error] Error exists in the bytecode
[error] Bytecode:
[error] 0000000: b1
[error] sbt.ForkMain$ForkError: java.lang.VerifyError: Constructor must call super() or this() before return
[error] Exception Details:
[error] Location:
[error] com/tsystems/toil/geolocation/model/RefinedTypes$UtmZone$.<init>()V @0: return
[error] Reason:
[error] Error exists in the bytecode
[error] Bytecode:
[error] 0000000: b1
[error]
[error] at com.tsystems.toil.geolocation.model.GeoPositionSpec$.<clinit>(GeoPositionSpec.scala:8)
[error] at java.base/jdk.internal.misc.Unsafe.ensureClassInitialized0(Native Method)Works without sbt coverage (i.e. sbt test only)
There is only one change that caused this to occur now:
Before (test runs were working but still got coverage error from this thread):
package com.tsystems.toil.geolocation.model
import eu.timepit.refined.*
import eu.timepit.refined.api.{ RefType, Refined, RefinedTypeOps }
import eu.timepit.refined.auto.*
import eu.timepit.refined.numeric.*
import eu.timepit.refined.collection.*
import eu.timepit.refined.numeric.Interval.*
import eu.timepit.refined.string.MatchesRegex
import eu.timepit.refined.types.numeric.*
import scala.language.adhocExtensions
trait RefinedTypes:
[...]
type UtmZone = Int Refined Closed[1, 60]
// provides an "unsafeFrom()" for UtmZone:
object UtmZone extends RefinedTypeOps[UtmZone, Int]
object RefinedTypes extends RefinedTypesAfter (test runs get the 'Error exists in the bytecode' from above):
package com.tsystems.toil.geolocation.model
import eu.timepit.refined.*
import eu.timepit.refined.api.{ RefType, Refined, RefinedTypeOps }
import eu.timepit.refined.auto.*
import eu.timepit.refined.numeric.*
import eu.timepit.refined.collection.*
import eu.timepit.refined.numeric.Interval.*
import eu.timepit.refined.string.MatchesRegex
import eu.timepit.refined.types.numeric.*
import scala.language.adhocExtensions
object RefinedTypes:
[...]
type UtmZone = Int Refined Closed[1, 60]
// provides an "unsafeFrom()" for UtmZone:
object UtmZone extends RefinedTypeOps[UtmZone, Int]So just remove the trait "RefinedTypes" and using it as an object directly.
longliveenduro
commented
1 year ago @TheElectronWill adjusted my example to reproduce the above "Error exists in bytecode". Please check master of https://github.com/longliveenduro/zio-geolocation-tapir-tapir-starter.
sbt clean test works. sbt clean coverage test coverageReport gives "Error exists in bytecode"
TheElectronWill
commented
1 year ago Thanks for the example! It will help me track the root cause of this issue.
TheElectronWill
commented
1 year ago After upgrading to scala 3.2.2-RC1 the tests run fine with coverage enabled. That makes sense because https://github.com/lampepfl/dotty/pull/16235 isn't actually part of 3.2.1 but of 3.2.2-RC1.
longliveenduro
commented
1 year ago Ah thank you very much. How do I find out in which release it lands/landed?
TheElectronWill
commented
1 year ago In the description of the Github release you'll find the list of improvements. https://github.com/lampepfl/dotty/releases/tag/3.2.2-RC1
longliveenduro
commented
1 year ago @TheElectronWill Just tested RC1. Works great, many thanks!
longliveenduro
commented
1 year ago Hi again @TheElectronWill,
just checked the overall coverage statement coverage of my ZIO2 + Scala 3 project. And I was surprised how low the number was. So i drilled into the HTML coverage reports and added a logging statement to make sure this part of the code is executed during the test.
Result is, that the coverage report shows it as untested, but I can see the log statement during the tests. Not sure what I should do next. Open a new bug? In this project? Currently this is closed source code, so I would have to put some effort in to create a reproducible public example. Please let me know if I should do that.
The whole for comprehension shows red (untested) in the HTML report:
case class GoogleMapsApiLive(context: GeoApiContext) extends GoogleMapsApi:
15 def call(payload: GeolocationPayload): IO[ProviderError, GeolocationResult] =
16 (for // red(untested) starts here
17 pendingResult <- ZIO.attempt(GeolocationApi.geolocate(context, payload)).mapError(toProviderError)
18 geolocationResult <- ZIO
19 .async[Any, ProviderError, GeolocationResult] { callback =>
20 pendingResult.setCallback(
21 new PendingResult.Callback[GeolocationResult]:
22 override def onResult(r: GeolocationResult) =
23 callback(ZIO.logError(">>>>>> GMaps API call onSuccess").zipRight(ZIO.succeed(r)))
24 override def onFailure(t: Throwable) = callback(ZIO.fail(toProviderError(t)))
25 )
26 }
27 yield geolocationResult) // red (untested) ends here
28 .catchAllDefect(t =>
29 ZIO.fail(toProviderError(t))
30 )But on the console (running the tests) I can see:
+ GoogleMapsApiSpec
timestamp=2022-12-01T19:13:54.275197577Z level=ERROR thread=#zio-fiber-943 message=">>>>>> GMaps API call onSuccess" location=com.tsystems.toil.geolocation.logic.google.GoogleMapsApiLive.call.$anon.onResult file=GoogleMapsApiLive.scala line=23Hi!
With code coverage instrumentation enabled, the compiler generates .scoverage files (usually somewhere in target/) for each source file. They contain information about the trees instrumented by the compiler. Here's an example from dotty's coverage tests:
instrumentCoverage)Does the for comprehension, or at least a part of it (like a method call to flatMap, ZIO.attempt, etc.) show up in your coverage files?
Depending on what the .scoverage file contains, we should open an issue either here or on Dotty.
I suspect that complex for ... yield might not be handled well, either because I've misimplemented the case in the instrumentCoverage phase, or because a previous compiler phase marks the tree as being "synthetic" (i.e. generated by the compiler and not the user - in which case it's normal not to instrument it).
longliveenduro
commented
1 year ago Thanks for your answer!
Unfortunately I don't get any hits if if do find . -name *.scoverage in the whole project directory (one dir above target/). I did run the test with coverage enabled directly before I did launch the find command. Also had a look around target, but could not find anything.
longliveenduro
commented
1 year ago Hi @TheElectronWill. I upated my example at https://github.com/longliveenduro/zio-geolocation-tapir-tapir-starter to use Scala 3.2.2-RC1. I also updated sbt, plugins and dependencies.
If you let the coverage report run on this example, there are no .scoverage files. Also if you check the html coverage report for EndpointsSpec, you can see that some statements are shown red/untested but the test definitely has to run over it:
E.g. endpoint.get and inside ZIO.succeed(s"Hello ${user.name}")). It almost looks like the colors are a bit off though.
Anyways, the test calls e.g. the "hello" endpoint and thus must go over the above "Hello ...".
TheElectronWill
commented
1 year ago I'll take a look, thanks!
If you compile with -coverage-out:/tmp/testcov it should produce coverage files in /tmp/testcov (I don't know where the scoverage plugin puts them by default, apparently not in target/ 😅).
ckipp01
commented
1 year ago apparently not in target/
No, this is where it should spit them. /target/scala-3.2.1/scoverage-data.
longliveenduro
commented
1 year ago @TheElectronWill @ckipp01 Thanks for the infos, but looking at these files I am a bit lost.
I see some problems with the coverage report for my example project https://github.com/longliveenduro/zio-geolocation-tapir-tapir-starter.
It is not only the for comprehension but e.g. also this section is marked red/untested:
endpoint.get
20 .in("hello")
21 .in(query[User]("name"))which is obviously covered by EndpointSpec.
rleibman
commented
1 year ago I got the same error, went to 3.2.2-RC1 and I no longer see the error. But It's no longer doing coverage properly, my project went from 86% to 49%. I think it's not aggregating the different modules the same.
nigeleke
commented
1 year ago Not sure if there's any relation to this issue: https://github.com/scoverage/sbt-scoverage/issues/382
The original issue is quite old - but I have added some examples below where coverage was expected but not reported.
longliveenduro
commented
1 year ago I tested Scala 3.2.2 and 3.3.0-RC2. Coverage plugin is working, but the numbers are not correct (too low).
I do not see that ZIO specs are instrumented: