search

com-lihaoyi / mill

Mill is a fast JVM build tool that supports Java and Scala. 2-4x faster than Gradle and 4-10x faster than Maven for common workflows, Mill aims to make your project’s build process performant, maintainable, and flexible
https://mill-build.org/
MIT License
2.2k stars 350 forks source link

Crash when trying to compile twirl templates? #407

Closed lihaoyi closed 6 years ago

lihaoyi commented 6 years ago 
lihaoyi test$ tree
.
├── build.sc
└── foo
    └── views
        └── foo.scala.html

2 directories, 2 files
lihaoyi test$ cat build.sc
import $ivy.`com.lihaoyi::mill-twirllib:0.2.6`
import mill._, scalalib._, twirllib._

object foo extends TwirlModule {
  def twirlVersion = "1.3.15"
}
lihaoyi test$ cat foo/views/foo.scala.html
(i: Int)
<h1>@i</h1>
lihaoyi test$  mill show foo.compileTwirl
[1/1] show
[4/4] foo.compileTwirl
1 targets failed
show 1 targets failed
foo.compileTwirl java.lang.reflect.InvocationTargetException
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:564)
    mill.twirllib.TwirlWorker$$anon$1.compileTwirl(TwirlWorker.scala:56)
    mill.twirllib.TwirlWorker.$anonfun$compile$2(TwirlWorker.scala:85)
    mill.twirllib.TwirlWorker.$anonfun$compile$2$adapted(TwirlWorker.scala:76)
    scala.collection.immutable.List.foreach(List.scala:389)
    mill.twirllib.TwirlWorker.compileTwirlDir$1(TwirlWorker.scala:76)
    mill.twirllib.TwirlWorker.$anonfun$compile$3(TwirlWorker.scala:90)
    mill.twirllib.TwirlWorker.$anonfun$compile$3$adapted(TwirlWorker.scala:90)
    scala.collection.Iterator.foreach(Iterator.scala:944)
    scala.collection.Iterator.foreach$(Iterator.scala:944)
    scala.collection.AbstractIterator.foreach(Iterator.scala:1432)
    scala.collection.IterableLike.foreach(IterableLike.scala:71)
    scala.collection.IterableLike.foreach$(IterableLike.scala:70)
    scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    mill.twirllib.TwirlWorker.compile(TwirlWorker.scala:90)
    mill.twirllib.TwirlModule.$anonfun$compileTwirl$2(TwirlModule.scala:45)
    mill.define.ApplyerGenerated.$anonfun$zipMap$1(ApplicativeGenerated.scala:5)
    mill.define.Task$MappedDest.evaluate(Task.scala:346)
java.lang.ExceptionInInitializerError
    play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$.getFunctionMapping(TwirlCompiler.scala:425)
    play.twirl.compiler.TwirlCompiler$.generateCode(TwirlCompiler.scala:326)
    play.twirl.compiler.TwirlCompiler$.generateFinalTemplate(TwirlCompiler.scala:377)
    play.twirl.compiler.TwirlCompiler$.parseAndGenerateCode(TwirlCompiler.scala:206)
    play.twirl.compiler.TwirlCompiler$.compile(TwirlCompiler.scala:174)
    play.twirl.compiler.TwirlCompiler.compile(TwirlCompiler.scala)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:564)
    mill.twirllib.TwirlWorker$$anon$1.compileTwirl(TwirlWorker.scala:56)
    mill.twirllib.TwirlWorker.$anonfun$compile$2(TwirlWorker.scala:85)
    mill.twirllib.TwirlWorker.$anonfun$compile$2$adapted(TwirlWorker.scala:76)
    scala.collection.immutable.List.foreach(List.scala:389)
    mill.twirllib.TwirlWorker.compileTwirlDir$1(TwirlWorker.scala:76)
    mill.twirllib.TwirlWorker.$anonfun$compile$3(TwirlWorker.scala:90)
    mill.twirllib.TwirlWorker.$anonfun$compile$3$adapted(TwirlWorker.scala:90)
    scala.collection.Iterator.foreach(Iterator.scala:944)
    scala.collection.Iterator.foreach$(Iterator.scala:944)
    scala.collection.AbstractIterator.foreach(Iterator.scala:1432)
    scala.collection.IterableLike.foreach(IterableLike.scala:71)
    scala.collection.IterableLike.foreach$(IterableLike.scala:70)
    scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    mill.twirllib.TwirlWorker.compile(TwirlWorker.scala:90)
    mill.twirllib.TwirlModule.$anonfun$compileTwirl$2(TwirlModule.scala:45)
    mill.define.ApplyerGenerated.$anonfun$zipMap$1(ApplicativeGenerated.scala:5)
    mill.define.Task$MappedDest.evaluate(Task.scala:346)
scala.reflect.internal.MissingRequirementError: object scala in compiler mirror not found.
    scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:17)
    scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:18)
    scala.reflect.internal.Mirrors$RootsBase.$anonfun$getModuleOrClass$4(Mirrors.scala:54)
    scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:54)
    scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
    scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackage$lzycompute(Definitions.scala:169)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackage(Definitions.scala:169)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackageClass$lzycompute(Definitions.scala:170)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackageClass(Definitions.scala:170)
    scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1424)
    scala.tools.nsc.Global$Run.<init>(Global.scala:1158)
    scala.tools.nsc.interactive.Global$TyperRun.<init>(Global.scala:1312)
    scala.tools.nsc.interactive.Global.newTyperRun(Global.scala:1335)
    scala.tools.nsc.interactive.Global.<init>(Global.scala:286)
    play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$CompilerInstance.compiler$lzycompute(TwirlCompiler.scala:507)
    play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$CompilerInstance.compiler(TwirlCompiler.scala:481)
    play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$PresentationCompiler$.<init>(TwirlCompiler.scala:549)
    play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$PresentationCompiler$.<clinit>(TwirlCompiler.scala)
    play.twirl.compiler.TwirlCompiler$TemplateAsFunctionCompiler$.getFunctionMapping(TwirlCompiler.scala:425)
    play.twirl.compiler.TwirlCompiler$.generateCode(TwirlCompiler.scala:326)
    play.twirl.compiler.TwirlCompiler$.generateFinalTemplate(TwirlCompiler.scala:377)
    play.twirl.compiler.TwirlCompiler$.parseAndGenerateCode(TwirlCompiler.scala:206)
    play.twirl.compiler.TwirlCompiler$.compile(TwirlCompiler.scala:174)
    play.twirl.compiler.TwirlCompiler.compile(TwirlCompiler.scala)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.base/java.lang.reflect.Method.invoke(Method.java:564)
    mill.twirllib.TwirlWorker$$anon$1.compileTwirl(TwirlWorker.scala:56)
    mill.twirllib.TwirlWorker.$anonfun$compile$2(TwirlWorker.scala:85)
    mill.twirllib.TwirlWorker.$anonfun$compile$2$adapted(TwirlWorker.scala:76)
    scala.collection.immutable.List.foreach(List.scala:389)
    mill.twirllib.TwirlWorker.compileTwirlDir$1(TwirlWorker.scala:76)
    mill.twirllib.TwirlWorker.$anonfun$compile$3(TwirlWorker.scala:90)
    mill.twirllib.TwirlWorker.$anonfun$compile$3$adapted(TwirlWorker.scala:90)
    scala.collection.Iterator.foreach(Iterator.scala:944)
    scala.collection.Iterator.foreach$(Iterator.scala:944)
    scala.collection.AbstractIterator.foreach(Iterator.scala:1432)
    scala.collection.IterableLike.foreach(IterableLike.scala:71)
    scala.collection.IterableLike.foreach$(IterableLike.scala:70)
    scala.collection.AbstractIterable.foreach(Iterable.scala:54)
    mill.twirllib.TwirlWorker.compile(TwirlWorker.scala:90)
    mill.twirllib.TwirlModule.$anonfun$compileTwirl$2(TwirlModule.scala:45)
    mill.define.ApplyerGenerated.$anonfun$zipMap$1(ApplicativeGenerated.scala:5)
    mill.define.Task$MappedDest.evaluate(Task.scala:346)
lihaoyi commented 6 years ago

@Mogztter any idea what this is?

alexarchambault commented 6 years ago

object scala in compiler mirror not found. might be indicative of a classpath issue. @lihaoyi If you have a test repo reproducing that issue, I'd be interested in checking if it's not a coursier issue…

lihaoyi commented 6 years ago

@alexarchambault the code block above shows the contents of all files and the command necessary to reproduce

ggrossetie commented 6 years ago

@lihaoyi Are you using Java8 ? Java9 ? I will try to reproduce this issue.

EDIT: I can reproduce this issue with JDK8

java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
ggrossetie commented 6 years ago

I think the Scala library is missing from the classloader used to load the class TwirlCompiler. https://github.com/playframework/twirl/blob/1d7c603352510809908d155393651748cc601e5e/compiler/src/main/scala/play/twirl/compiler/TwirlCompiler.scala#L507

At this line, we can see that the TwirlCompiler is using scala.tools.nsc.interactive.Global.

ggrossetie commented 6 years ago

I tried to add the following dependency in the twirlClasspath function but without success:

ivy"org.scala-lang:scala-compiler:2.12.4"

I've built a local version of mill using the procedure described in the README and I've noticed that the dependency is mill-contrib-twirllib instead of mill-twirllib:

import $ivy.`com.lihaoyi::mill-contrib-twirllib:0.2.6-16-6220c7-DIRTY85e9bfb4`

Anyway it's not working. Some people on Stackoverflow mention the settings settings.usejavacp.value = true but I'm a bit skeptical...

ggrossetie commented 6 years ago

I've noticed that the TwirlCompiler class is using the following code to locate classes/sources code:

Class.forName("scala.Predef").getProtectionDomain.getCodeSource

When running the test suite the return value is something like file:/path/to/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/org/scala-lang/scala-library/2.12.6/scala-library-2.12.6.jar <no signer certificates>.

But when running the mill command the return value is file:/path/to/mill-release <no signer certificates>.

It may explain why the presentation compiler instantiated in the TwirlCompiler can't find the Scala library.

I don't really know why the TwirlCompiler is using this strategy and how we can resolve this issue... Any idea @lihaoyi @rockjam ? For reference:

https://github.com/playframework/twirl/blob/1d7c603352510809908d155393651748cc601e5e/compiler/src/main/scala/play/twirl/compiler/TwirlCompiler.scala#L485-L505

alexarchambault commented 6 years ago

If that's of any help, as a workaround, running mill via coursier fixes that

$ coursier launch com.lihaoyi:mill-dev_2.12:0.2.4 -M mill.MillMain -- \
    show foo.compileTwirl
Compiling (synthetic)/ammonite/predef/interpBridge.sc
Compiling (synthetic)/ammonite/predef/DefaultPredef.sc
Compiling …/build.sc
[1/1] show
[4/4] foo.compileTwirl
{
    "analysisFile": "…/out/foo/compileTwirl/dest/zinc",
    "classes": "ref:586a1b4f:…/out/foo/compileTwirl/dest/html"
}

(because coursier launch relies on standard jars, like in the tests, rather than a fat jar)

lihaoyi commented 6 years ago

Fixed in https://github.com/lihaoyi/mill/commit/b29c13d9010f9405de04c471fc4c8ba816606fbc