search

com-lihaoyi / mill

Mill is a fast JVM build tool that supports Java and Scala. 2-4x faster than Gradle and 4-10x faster than Maven for common workflows, Mill aims to make your project’s build process performant, maintainable, and flexible
https://mill-build.org/
MIT License
2.21k stars 353 forks source link

Excluding Scala library from assembly crashes compile task #818

Closed martingd closed 3 years ago

martingd commented 4 years ago

According to the documentation, the Scala library can be excluded from the assembly like this:

def scalaLibraryIvyDeps = T { Agg.empty }

However, this causes a crash in the compile task.

Example

This example shows the issue using Mill 0.6.1 on macOS 10.15 Catalina with JDK 1.8.0_231:

% mill --version
Mill Build Tool version 0.6.1
Java version: 1.8.0_231, vendor: Oracle Corporation, runtime: /Library/Java/JavaVirtualMachines/jdk1.8.0_231.jdk/Contents/Home/jre
Default locale: en_GB, platform encoding: UTF-8
OS name: "Mac OS X", version: 10.15.4, arch: x86_64

Project Layout

% tree .
.
├── build.sc
└── main
    └── src
        └── org
            └── example
                └── Main.scala

Mill build script

import mill._, scalalib._

object main extends ScalaModule {
    def scalaVersion = "2.12.4"
    def scalaLibraryIvyDeps = T { Agg.empty }
}

Also tested with scalaVersion set to "2.10.0" and "2.13.1".

Failing compile task

% mill main.compile
[25/25] main.compile 
Compiling compiler interface...
warning: there were three deprecation warnings (since 2.12.0); re-run with -deprecation for details
warning: there were three feature warnings; re-run with -feature for details
two warnings found
[info] Compiling 1 Scala source to /Users/johndoe/tmp/mill-exclude-issue/out/main/compile/dest/classes ...
[error] error while loading Object, Missing dependency 'object scala in compiler mirror', required by /Library/Java/JavaVirtualMachines/jdk1.8.0_231.jdk/Contents/Home/jre/lib/rt.jar(java/lang/Object.class)
[error] ## Exception when compiling 1 sources to /Users/johndoe/tmp/mill-exclude-issue/out/main/compile/dest/classes
[error] object scala in compiler mirror not found.
[error] scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:17)
[error] scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:18)
[error] scala.reflect.internal.Mirrors$RootsBase.$anonfun$getModuleOrClass$4(Mirrors.scala:54)
[error] scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:54)
[error] scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
[error] scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
[error] scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackage$lzycompute(Definitions.scala:169)
[error] scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackage(Definitions.scala:169)
[error] scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackageClass$lzycompute(Definitions.scala:170)
[error] scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackageClass(Definitions.scala:170)
[error] scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1423)
[error] scala.tools.nsc.Global$Run.<init>(Global.scala:1164)
[error] xsbt.ZincCompiler$ZincRun.<init>(CallbackGlobal.scala:63)
[error] xsbt.CachedCompiler0.run(CompilerInterface.scala:128)
[error] xsbt.CachedCompiler0.run(CompilerInterface.scala:105)
[error] xsbt.CompilerInterface.run(CompilerInterface.scala:31)
[error] sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
[error] sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
[error] sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
[error] java.lang.reflect.Method.invoke(Method.java:498)
[error] sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:237)
[error] sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:111)
[error] sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:90)
[error] sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:82)
[error] scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
[error] sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:133)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:73)
[error] sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:116)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:307)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:307)
[error] sbt.internal.inc.Incremental$.doCompile(Incremental.scala:106)
[error] sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:87)
[error] sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:116)
[error] sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:63)
[error] sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:89)
[error] sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:134)
[error] sbt.internal.inc.Incremental$.compile(Incremental.scala:80)
[error] sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:67)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:311)
[error] sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:269)
[error] sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:159)
[error] sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:238)
[error] sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:69)
[error] mill.scalalib.worker.ZincWorkerImpl.compileInternal(ZincWorkerImpl.scala:433)
[error] mill.scalalib.worker.ZincWorkerImpl.$anonfun$compileMixed0$1(ZincWorkerImpl.scala:270)
[error] mill.api.KeyedLockedCache$RandomBoundedCache.withCachedValue(KeyedLockedCache.scala:31)
[error] mill.scalalib.worker.ZincWorkerImpl.withCompilers(ZincWorkerImpl.scala:341)
[error] mill.scalalib.worker.ZincWorkerImpl.compileMixed0(ZincWorkerImpl.scala:269)
[error] mill.scalalib.worker.ZincWorkerImpl.compileMixed(ZincWorkerImpl.scala:239)
[error] mill.scalalib.ScalaModule.$anonfun$compile$2(ScalaModule.scala:143)
[error] mill.define.ApplyerGenerated.$anonfun$zipMap$9(ApplicativeGenerated.scala:21)
[error] mill.define.Task$MappedDest.evaluate(Task.scala:379)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$14(Evaluator.scala:343)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] scala.Console$.withErr(Console.scala:196)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$13(Evaluator.scala:343)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] scala.Console$.withOut(Console.scala:167)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$12(Evaluator.scala:342)
[error] scala.util.DynamicVariable.withValue(DynamicVariable.scala:62)
[error] scala.Console$.withIn(Console.scala:230)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$8(Evaluator.scala:341)
[error] mill.eval.Evaluator.$anonfun$evaluateGroup$8$adapted(Evaluator.scala:295)
[error] scala.collection.Iterator.foreach(Iterator.scala:941)
[error] scala.collection.Iterator.foreach$(Iterator.scala:941)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
[error] scala.collection.IterableLike.foreach(IterableLike.scala:74)
[error] scala.collection.IterableLike.foreach$(IterableLike.scala:73)
[error] scala.collection.AbstractIterable.foreach(Iterable.scala:56)
[error] mill.eval.Evaluator.evaluateGroup(Evaluator.scala:295)
[error] mill.eval.Evaluator.evaluateGroupCached(Evaluator.scala:200)
[error] mill.eval.Evaluator.$anonfun$evaluate$2(Evaluator.scala:80)
[error] mill.eval.Evaluator.$anonfun$evaluate$2$adapted(Evaluator.scala:61)
[error] scala.collection.Iterator.foreach(Iterator.scala:941)
[error] scala.collection.Iterator.foreach$(Iterator.scala:941)
[error] scala.collection.AbstractIterator.foreach(Iterator.scala:1429)
[error] mill.eval.Evaluator.evaluate(Evaluator.scala:61)
[error] mill.main.RunScript$.evaluate(RunScript.scala:208)
[error] mill.main.RunScript$.$anonfun$evaluateTasks$1(RunScript.scala:194)
[error] scala.util.Either.map(Either.scala:353)
[error] mill.main.RunScript$.evaluateTasks(RunScript.scala:193)
[error] mill.main.RunScript$.$anonfun$runScript$4(RunScript.scala:67)
[error] ammonite.util.Res$Success.flatMap(Res.scala:62)
[error] mill.main.RunScript$.runScript(RunScript.scala:66)
[error] mill.main.MainRunner.$anonfun$runScript$1(MainRunner.scala:94)
[error] mill.main.MainRunner.watchLoop2(MainRunner.scala:57)
[error] mill.main.MainRunner.runScript(MainRunner.scala:71)
[error] mill.MillMain$.main0(MillMain.scala:213)
[error] mill.main.MillServerMain$.main0(MillServerMain.scala:67)
[error] mill.main.Server.$anonfun$handleRun$1(MillServerMain.scala:154)
[error] java.lang.Thread.run(Thread.java:748)
[error]            
1 targets failed
main.compile scala.reflect.internal.MissingRequirementError: object scala in compiler mirror not found.
    scala.reflect.internal.MissingRequirementError$.signal(MissingRequirementError.scala:17)
    scala.reflect.internal.MissingRequirementError$.notFound(MissingRequirementError.scala:18)
    scala.reflect.internal.Mirrors$RootsBase.$anonfun$getModuleOrClass$4(Mirrors.scala:54)
    scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:54)
    scala.reflect.internal.Mirrors$RootsBase.getModuleOrClass(Mirrors.scala:66)
    scala.reflect.internal.Mirrors$RootsBase.getPackage(Mirrors.scala:172)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackage$lzycompute(Definitions.scala:169)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackage(Definitions.scala:169)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackageClass$lzycompute(Definitions.scala:170)
    scala.reflect.internal.Definitions$DefinitionsClass.ScalaPackageClass(Definitions.scala:170)
    scala.reflect.internal.Definitions$DefinitionsClass.init(Definitions.scala:1423)
    scala.tools.nsc.Global$Run.<init>(Global.scala:1164)
    xsbt.ZincCompiler$ZincRun.<init>(CallbackGlobal.scala:63)
    xsbt.CachedCompiler0.run(CompilerInterface.scala:128)
    xsbt.CachedCompiler0.run(CompilerInterface.scala:105)
    xsbt.CompilerInterface.run(CompilerInterface.scala:31)
    sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    java.lang.reflect.Method.invoke(Method.java:498)
    sbt.internal.inc.AnalyzingCompiler.call(AnalyzingCompiler.scala:237)
    sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:111)
    sbt.internal.inc.AnalyzingCompiler.compile(AnalyzingCompiler.scala:90)
    sbt.internal.inc.MixedAnalyzingCompiler.$anonfun$compile$3(MixedAnalyzingCompiler.scala:82)
    scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.java:23)
    sbt.internal.inc.MixedAnalyzingCompiler.timed(MixedAnalyzingCompiler.scala:133)
    sbt.internal.inc.MixedAnalyzingCompiler.compileScala$1(MixedAnalyzingCompiler.scala:73)
    sbt.internal.inc.MixedAnalyzingCompiler.compile(MixedAnalyzingCompiler.scala:116)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1(IncrementalCompilerImpl.scala:307)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileInternal$1$adapted(IncrementalCompilerImpl.scala:307)
    sbt.internal.inc.Incremental$.doCompile(Incremental.scala:106)
    sbt.internal.inc.Incremental$.$anonfun$compile$4(Incremental.scala:87)
    sbt.internal.inc.IncrementalCommon.recompileClasses(IncrementalCommon.scala:116)
    sbt.internal.inc.IncrementalCommon.cycle(IncrementalCommon.scala:63)
    sbt.internal.inc.Incremental$.$anonfun$compile$3(Incremental.scala:89)
    sbt.internal.inc.Incremental$.manageClassfiles(Incremental.scala:134)
    sbt.internal.inc.Incremental$.compile(Incremental.scala:80)
    sbt.internal.inc.IncrementalCompile$.apply(Compile.scala:67)
    sbt.internal.inc.IncrementalCompilerImpl.compileInternal(IncrementalCompilerImpl.scala:311)
    sbt.internal.inc.IncrementalCompilerImpl.$anonfun$compileIncrementally$1(IncrementalCompilerImpl.scala:269)
    sbt.internal.inc.IncrementalCompilerImpl.handleCompilationError(IncrementalCompilerImpl.scala:159)
    sbt.internal.inc.IncrementalCompilerImpl.compileIncrementally(IncrementalCompilerImpl.scala:238)
    sbt.internal.inc.IncrementalCompilerImpl.compile(IncrementalCompilerImpl.scala:69)
    mill.scalalib.worker.ZincWorkerImpl.compileInternal(ZincWorkerImpl.scala:433)
    mill.scalalib.worker.ZincWorkerImpl.$anonfun$compileMixed0$1(ZincWorkerImpl.scala:270)
    mill.api.KeyedLockedCache$RandomBoundedCache.withCachedValue(KeyedLockedCache.scala:31)
    mill.scalalib.worker.ZincWorkerImpl.withCompilers(ZincWorkerImpl.scala:341)
    mill.scalalib.worker.ZincWorkerImpl.compileMixed0(ZincWorkerImpl.scala:269)
    mill.scalalib.worker.ZincWorkerImpl.compileMixed(ZincWorkerImpl.scala:239)
    mill.scalalib.ScalaModule.$anonfun$compile$2(ScalaModule.scala:143)
    mill.define.ApplyerGenerated.$anonfun$zipMap$9(ApplicativeGenerated.scala:21)
    mill.define.Task$MappedDest.evaluate(Task.scala:379)
%
martingd commented 4 years ago

I am aware there is a workaround:

def assemblyRules = Assembly.defaultRules ++ Seq(ExcludePattern("scala/.*"))

as described in this comment to issue #575.

However, the docs states that the above method should work and is the preferred method of excluding the Scala library from the assembly.

lefou commented 4 years ago

Thank for reporting! Can't tell why it isn't working right now though. Will look into it.

joan38 commented 4 years ago

This issue seem due to transitive dependencies pulling the Scala lib.

lefou commented 4 years ago

The scalaLibraryIvyDeps target is also used in the compileClasspath to compile the code. So, if you make it empty and provide the scala library by no other means, than the compilation fails as expected. One way to fix this is to add the scala library via compileIvyDeps.

lefou commented 3 years ago

The documentation was wrong. We also changed scala libaray handling in mill. I fixed the documentation in #1507. This issue is no longer valid. See documentation for how to properly exclude files from assembly.