search

sbt / sbt-assembly

Deploy über-JARs. Restart processes. (port of codahale/assembly-sbt)
MIT License
1.95k stars 224 forks source link

Duplicate file exception occurs when packing graal vm #370

Closed ZhiruiLi closed 5 years ago

ZhiruiLi commented 5 years ago

Hi, I trying to use graal vm in my Scala project as JS engine. This project will finally be packed as one fat jar file by sbt assembly. When I run project in IDEA, everything goes well, but when I try to pack it using sbt assembly, file duplicate exception occurs:

If the project only add dependency of "org.graalvm.js" % "js-scriptengine" % "1.0.0-rc14", the error will be (ignore many output in ... ):

[error] java.lang.RuntimeException: deduplicate: different file contents found in the following:
[error] /Users/zhiruili/.ivy2/cache/org.graalvm.js/js-scriptengine/jars/js-scriptengine-1.0.0-rc14.jar:module-info.class
[error] /Users/zhiruili/.ivy2/cache/org.graalvm.sdk/graal-sdk/jars/graal-sdk-1.0.0-rc14.jar:module-info.class
......

If the project only add dependency of

"org.graalvm.js" % "js" % "1.0.0-rc14",
"org.graalvm.sdk" % "graal-sdk" % "1.0.0-rc14",
"org.graalvm.truffle" % "truffle-api" % "1.0.0-rc14",

the error will be (ignore many output in ... ):

[error] deduplicate: different file contents found in the following:
[error] /Users/zhiruili/.ivy2/cache/org.graalvm.js/js/jars/js-1.0.0-rc14.jar:module-info.java
[error] /Users/zhiruili/.ivy2/cache/org.graalvm.regex/regex/jars/regex-1.0.0-rc14.jar:module-info.java
[error] /Users/zhiruili/.ivy2/cache/org.graalvm.truffle/truffle-api/jars/truffle-api-1.0.0-rc14.jar:module-info.java
[error] /Users/zhiruili/.ivy2/cache/org.graalvm.sdk/graal-sdk/jars/graal-sdk-1.0.0-rc14.jar:module-info.java
[error]         at sbtassembly.Assembly$.applyStrategies(Assembly.scala:143)
...

Can anyone know how to solve this problem?

voidconductor commented 5 years ago

You need to set merge strategy for these dependencies

assembly / assemblyMergeStrategy := {
  case PathList("module-info.java") => MergeStrategy.discard
  case PathList("module-info.class") => MergeStrategy.discard
  case "META-INF/truffle/language" => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

This worked for my case with java 8 and Graal 19.0.0

ZhiruiLi commented 5 years ago

Thanks！

voidconductor commented 5 years ago

I had to change "META-INF/truffle/language" => MergeStrategy.discard to "META-INF/truffle/language" => MergeStrategy.rename

I also had to create resources/META-INF/truffle/language file containing

language1.charactermimetype.0=application/javascript
language1.charactermimetype.1=application/javascript+module
language1.charactermimetype.2=text/javascript
language1.classname=com.oracle.truffle.js.lang.javascriptlanguage
language1.defaultmimetype=application/javascript
language1.dependentlanguage.0=regex
language1.filetypedetector0=com.oracle.truffle.js.lang.jsfiletypedetector
language1.id=js
language1.implementationname=graalvm javascript
language1.interactive=true
language1.internal=false
language1.name=javascript
language1.version=inherit

language2.charactermimetype.0=application/tregex
language2.classname=com.oracle.truffle.regex.regexlanguage
language2.id=regex
language2.implementationname=
language2.interactive=false
language2.internal=true
language2.name=regex
language2.version=0.1

otherwise library couldn't find languages

deekayman commented 3 years ago

You need to set merge strategy for these dependencies

assembly / assemblyMergeStrategy := {
  case PathList("module-info.java") => MergeStrategy.discard
  case PathList("module-info.class") => MergeStrategy.discard
  case "META-INF/truffle/language" => MergeStrategy.discard
  case x =>
    val oldStrategy = (assemblyMergeStrategy in assembly).value
    oldStrategy(x)
}

This worked for my case with java 8 and Graal 19.0.0

This does not work.

[error] 1 error was encountered during merge
[error] 1 error was encountered during merge
[error] 1 error was encountered during merge
[info] Strategy 'discard' was applied to 48 files (Run the task at debug level to see details)
[info] Strategy 'rename' was applied to 7 files (Run the task at debug level to see details)
[error] java.lang.RuntimeException: deduplicate: different file contents found in the following:
[error] /Users/dave/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.12.0/jackson-annotations-2.12.0.jar:module-info.class
[error] /Users/dave/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.12.0/jackson-core-2.12.0.jar:module-info.class
[error] /Users/dave/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.12.0/jackson-databind-2.12.0.jar:module-info.class
voidconductor commented 3 years ago

This does not work.

[error] /Users/dave/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.12.0/jackson-annotations-2.12.0.jar:module-info.class
[error] /Users/dave/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-core/2.12.0/jackson-core-2.12.0.jar:module-info.class
[error] /Users/dave/Library/Caches/Coursier/v1/https/repo1.maven.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.12.0/jackson-databind-2.12.0.jar:module-info.class

Looks like it's not related to graalvm