search

sbt / sbt-assembly

Deploy über-JARs. Restart processes. (port of codahale/assembly-sbt)
MIT License
1.95k stars 224 forks source link

Every class is wrongly found duplicate times when working with SNAPSHOT library ("Deduplicate found different file contents") #529

Open Sciss opened 3 months ago

Sciss commented 3 months ago

I'm working on locally published SNAPSHOT minor version of a library, that when I use it in the target (local) project, produces new assembly errors, basically like this:

[info] compiling 1 Scala source to /home/hhrutz/Documents/devel/Mellite/app/target/scala-2.13/classes ...
[error] 1139 error(s) were encountered during the merge:
[error] java.lang.RuntimeException: 
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/SchedulerImpl$Impl.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/SchedulerImpl$Impl.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/SchedulerImpl$Impl.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/AuralSystemImpl$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/AuralSystemImpl$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/AuralSystemImpl$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Timeline$Remove$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Timeline$Remove$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Timeline$Remove$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/AuralAttributeImpl$ExprLikeWithRef.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/AuralAttributeImpl$ExprLikeWithRef.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/AuralAttributeImpl$ExprLikeWithRef.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/UGenGraphBuilder$Input$DiskOut.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/UGenGraphBuilder$Input$DiskOut.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/UGenGraphBuilder$Input$DiskOut.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/EnvSegment$Multi.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/EnvSegment$Multi.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/EnvSegment$Multi.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/FadeSpec$Curve.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/FadeSpec$Curve.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/FadeSpec$Curve.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = lucre-synth_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/synth/Buffer$.class
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/synth/Buffer$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/synth/Buffer$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/synth/Buffer$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/MidiBase$Device$IsOutput$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/MidiBase$Device$IsOutput$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/MidiBase$Device$IsOutput$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Runner$ExpandedMessages.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Runner$ExpandedMessages.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Runner$ExpandedMessages.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Color$MixOp$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Color$MixOp$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Color$MixOp$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/ThisRunner$ExpandedFail.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/ThisRunner$ExpandedFail.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/ThisRunner$ExpandedFail.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Color$FromHSB$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Color$FromHSB$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Color$FromHSB$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/Code$Program$CodeImpl$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/Code$Program$CodeImpl$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/Code$Program$CodeImpl$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/RunnerUniverseImpl$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/RunnerUniverseImpl$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/RunnerUniverseImpl$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Runner$RunWith$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Runner$RunWith$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Runner$RunWith$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/WidgetImpl$Impl$changed$.class
[error]   Jar name = soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/proc/impl/WidgetImpl$Impl$changed$.class
[error] Deduplicate found different file contents in the following:
[error]   Jar name = soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Proc$Ops$.class
[error]   Jar name = soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, jar org = de.sciss, entry target = de/sciss/lucre/expr/graph/Proc$Ops$.class
...

Basically every class is found in multiple files, although in reality they are not in multiple files. For example, de/sciss/proc/impl/SchedulerImpl$Impl.class is only in soundprocesses-core_2.13-4.14.13-SNAPSHOT.jar, but not in soundprocesses-compiler_2.13-4.14.13-SNAPSHOT.jar or soundprocesses-views_2.13-4.14.13-SNAPSHOT.jar (thus falsely reported). I sometimes made it work by bumping the minor version further when updating, but now it doesn't work either. Seems to be a very nasty cache problem. I already removed target etc., but probably the cache is elsewhere. I also tried

ThisBuild / assemblyCacheOutput := false

but has no effect.

Sciss commented 3 months ago

Wait. Sorry. The jars do contain the classes multiple times. WTF.

$ jar -tf /home/hhrutz/.ivy2/local/de.sciss/soundprocesses-core_2.13/4.14.14-SNAPSHOT/jars/soundprocesses-core_2.13.jar | grep SchedulerImpl\$Impl
de/sciss/proc/impl/SchedulerImpl$Impl.class
$ jar -tf /home/hhrutz/.ivy2/local/de.sciss/soundprocesses-compiler_2.13/4.14.14-SNAPSHOT/jars/soundprocesses-compiler_2.13.jar | grep SchedulerImpl\$Impl
de/sciss/proc/impl/SchedulerImpl$Impl.class

Perhaps a crossproject bug? Anyway, seems unrelated to sbt-assembly, closing.