Dependency conflicts on org.ow2.asm:asm, leading to inconsistent program behaviors

Hi, in webmagic-WebMagic-0.7.3/webmagic-scripts, there are mulptiple versions of library org.ow2.asm:asm. However, according to Maven's dependency management strategy: "first declaration wins", only org.ow2.asm:asm:4.0 can be loaded, and org.ow2.asm:asm:5.0.4 will be shadowed.

In total, there are 10 conflicting API pairs between these two library version.

As shown in the following figure, your project expects to invoke method <org.objectweb.asm.ClassWriter: toByteArray()[B> in library org.ow2.asm:asm:5.0.4 (along the original dependency path). As it has been shadowed, this method defined in org.ow2.asm:asm:4.0 is actually forced to be referenced via the following invocation path (along the actual dependency path):

<us.codecraft.webmagic.scripts.ScriptConsole: main([Ljava/lang/String;)V> /home/wwww/sensor/unzip/webmagic-WebMagic-0.7.3/webmagic-scripts/target/classes
<us.codecraft.webmagic.scripts.ScriptConsole: startSpider(Lus/codecraft/webmagic/scripts/ScriptConsole$Params;)V> /home/wwww/sensor/unzip/webmagic-WebMagic-0.7.3/webmagic-scripts/target/classes
<us.codecraft.webmagic.Spider: run()V> /home/wwww/.m2/repository/us/codecraft/webmagic-core/0.7.3/webmagic-core-0.7.3.jar
<us.codecraft.webmagic.Spider$1: run()V> /home/wwww/.m2/repository/us/codecraft/webmagic-core/0.7.3/webmagic-core-0.7.3.jar
<us.codecraft.webmagic.Spider: access$000(Lus/codecraft/webmagic/Spider;Lus/codecraft/webmagic/Request;)V> /home/wwww/.m2/repository/us/codecraft/webmagic-core/0.7.3/webmagic-core-0.7.3.jar
<us.codecraft.webmagic.Spider: processRequest(Lus/codecraft/webmagic/Request;)V> /home/wwww/.m2/repository/us/codecraft/webmagic-core/0.7.3/webmagic-core-0.7.3.jar
<us.codecraft.webmagic.Spider: onDownloadSuccess(Lus/codecraft/webmagic/Request;Lus/codecraft/webmagic/Page;)V> /home/wwww/.m2/repository/us/codecraft/webmagic-core/0.7.3/webmagic-core-0.7.3.jar
<us.codecraft.webmagic.scripts.ScriptProcessor: process(Lus/codecraft/webmagic/Page;)V> /home/wwww/sensor/unzip/webmagic-WebMagic-0.7.3/webmagic-scripts/target/classes
<net.minidev.json.JSONObject: toString()Ljava/lang/String;> /home/wwww/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar
<net.minidev.json.JSONObject: toJSONString(Ljava/util/Map;Lnet/minidev/json/JSONStyle;)Ljava/lang/String;> /home/wwww/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar
<net.minidev.json.JSONObject: writeJSON(Ljava/util/Map;Ljava/lang/Appendable;Lnet/minidev/json/JSONStyle;)V> /home/wwww/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar
<net.minidev.json.reader.BeansWriterASM: writeJSONString(Ljava/lang/Object;Ljava/lang/Appendable;Lnet/minidev/json/JSONStyle;)V> /home/wwww/.m2/repository/net/minidev/json-smart/2.3/json-smart-2.3.jar
<net.minidev.asm.BeansAccess: get(Ljava/lang/Class;Lnet/minidev/asm/FieldFilter;)Lnet/minidev/asm/BeansAccess;> /home/wwww/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar
<net.minidev.asm.BeansAccessBuilder: bulid()Ljava/lang/Class;> /home/wwww/.m2/repository/net/minidev/accessors-smart/1.2/accessors-smart-1.2.jar
<org.objectweb.asm.ClassWriter: toByteArray()[B>

webmagic-WebMagic

Although both of these conflicting libraries contain the referenced methods (with the same signature), they have different implementations. This issue will not cause runtime crashes, but it can introduce inconsistent semantic program hehaviors----

Code snippet of <org.objectweb.asm.ClassWriter: toByteArray()[B> in org.ow2.asm:asm:5.0.4 (shadowed but expected to invoke method):

detailed method body

```java public byte[] toByteArray() { if (this.c > 65535) { throw new RuntimeException("Class file too large!"); } else { int var1 = 24 + 2 * this.o; int var2 = 0; FieldWriter var3; for(var3 = this.B; var3 != null; var3 = (FieldWriter)var3.fv) { ++var2; var1 += var3.a(); } int var4 = 0; MethodWriter var5; for(var5 = this.D; var5 != null; var5 = (MethodWriter)var5.mv) { ++var4; var1 += var5.a(); } int var6 = 0; if (this.A != null) { ++var6; var1 += 8 + this.A.b; this.newUTF8("BootstrapMethods"); } if (this.m != 0) { ++var6; var1 += 8; this.newUTF8("Signature"); } if (this.q != 0) { ++var6; var1 += 8; this.newUTF8("SourceFile"); } if (this.r != null) { ++var6; var1 += this.r.b + 6; this.newUTF8("SourceDebugExtension"); } if (this.s != 0) { ++var6; var1 += 10; this.newUTF8("EnclosingMethod"); } if ((this.k & 131072) != 0) { ++var6; var1 += 6; this.newUTF8("Deprecated"); } if ((this.k & 4096) != 0 && ((this.b & '\uffff') < 49 || (this.k & 262144) != 0)) { ++var6; var1 += 6; this.newUTF8("Synthetic"); } if (this.y != null) { ++var6; var1 += 8 + this.y.b; this.newUTF8("InnerClasses"); } if (this.u != null) { ++var6; var1 += 8 + this.u.a(); this.newUTF8("RuntimeVisibleAnnotations"); } if (this.v != null) { ++var6; var1 += 8 + this.v.a(); this.newUTF8("RuntimeInvisibleAnnotations"); } if (this.N != null) { ++var6; var1 += 8 + this.N.a(); this.newUTF8("RuntimeVisibleTypeAnnotations"); } if (this.O != null) { ++var6; var1 += 8 + this.O.a(); this.newUTF8("RuntimeInvisibleTypeAnnotations"); } if (this.w != null) { var6 += this.w.a(); var1 += this.w.a(this, (byte[])null, 0, -1, -1); } var1 += this.d.b; ByteVector var7 = new ByteVector(var1); var7.putInt(-889275714).putInt(this.b); var7.putShort(this.c).putByteArray(this.d.a, 0, this.d.b); int var8 = 393216 | (this.k & 262144) / 64; var7.putShort(this.k & ~var8).putShort(this.l).putShort(this.n); var7.putShort(this.o); int var9; for(var9 = 0; var9 < this.o; ++var9) { var7.putShort(this.p[var9]); } var7.putShort(var2); for(var3 = this.B; var3 != null; var3 = (FieldWriter)var3.fv) { var3.a(var7); } var7.putShort(var4); for(var5 = this.D; var5 != null; var5 = (MethodWriter)var5.mv) { var5.a(var7); } var7.putShort(var6); if (this.A != null) { var7.putShort(this.newUTF8("BootstrapMethods")); var7.putInt(this.A.b + 2).putShort(this.z); var7.putByteArray(this.A.a, 0, this.A.b); } if (this.m != 0) { var7.putShort(this.newUTF8("Signature")).putInt(2).putShort(this.m); } if (this.q != 0) { var7.putShort(this.newUTF8("SourceFile")).putInt(2).putShort(this.q); } if (this.r != null) { var9 = this.r.b; var7.putShort(this.newUTF8("SourceDebugExtension")).putInt(var9); var7.putByteArray(this.r.a, 0, var9); } if (this.s != 0) { var7.putShort(this.newUTF8("EnclosingMethod")).putInt(4); var7.putShort(this.s).putShort(this.t); } if ((this.k & 131072) != 0) { var7.putShort(this.newUTF8("Deprecated")).putInt(0); } if ((this.k & 4096) != 0 && ((this.b & '\uffff') < 49 || (this.k & 262144) != 0)) { var7.putShort(this.newUTF8("Synthetic")).putInt(0); } if (this.y != null) { var7.putShort(this.newUTF8("InnerClasses")); var7.putInt(this.y.b + 2).putShort(this.x); var7.putByteArray(this.y.a, 0, this.y.b); } if (this.u != null) { var7.putShort(this.newUTF8("RuntimeVisibleAnnotations")); this.u.a(var7); } if (this.v != null) { var7.putShort(this.newUTF8("RuntimeInvisibleAnnotations")); this.v.a(var7); } if (this.N != null) { var7.putShort(this.newUTF8("RuntimeVisibleTypeAnnotations")); this.N.a(var7); } if (this.O != null) { var7.putShort(this.newUTF8("RuntimeInvisibleTypeAnnotations")); this.O.a(var7); } if (this.w != null) { this.w.a(this, (byte[])null, 0, -1, -1, var7); } if (this.L) { this.u = null; this.v = null; this.w = null; this.x = 0; this.y = null; this.z = 0; this.A = null; this.B = null; this.C = null; this.D = null; this.E = null; this.K = false; this.J = true; this.L = false; (new ClassReader(var7.a)).accept(this, 4); return this.toByteArray(); } else { return var7.a; } } } ```

Code snippet of <org.objectweb.asm.ClassWriter: toByteArray()[B> in org.ow2.asm:asm:4.0 (loaded version):

detailed method body

```java public byte[] toByteArray() { if (this.c > 32767) { throw new RuntimeException("Class file too large!"); } else { int var1 = 24 + 2 * this.o; int var2 = 0; FieldWriter var3; for(var3 = this.B; var3 != null; var3 = (FieldWriter)var3.fv) { ++var2; var1 += var3.a(); } int var4 = 0; MethodWriter var5; for(var5 = this.D; var5 != null; var5 = (MethodWriter)var5.mv) { ++var4; var1 += var5.a(); } int var6 = 0; if (this.A != null) { ++var6; var1 += 8 + this.A.b; this.newUTF8("BootstrapMethods"); } if (this.m != 0) { ++var6; var1 += 8; this.newUTF8("Signature"); } if (this.q != 0) { ++var6; var1 += 8; this.newUTF8("SourceFile"); } if (this.r != null) { ++var6; var1 += this.r.b + 4; this.newUTF8("SourceDebugExtension"); } if (this.s != 0) { ++var6; var1 += 10; this.newUTF8("EnclosingMethod"); } if ((this.k & 131072) != 0) { ++var6; var1 += 6; this.newUTF8("Deprecated"); } if ((this.k & 4096) != 0 && ((this.b & '\uffff') < 49 || (this.k & 262144) != 0)) { ++var6; var1 += 6; this.newUTF8("Synthetic"); } if (this.y != null) { ++var6; var1 += 8 + this.y.b; this.newUTF8("InnerClasses"); } if (this.u != null) { ++var6; var1 += 8 + this.u.a(); this.newUTF8("RuntimeVisibleAnnotations"); } if (this.v != null) { ++var6; var1 += 8 + this.v.a(); this.newUTF8("RuntimeInvisibleAnnotations"); } if (this.w != null) { var6 += this.w.a(); var1 += this.w.a(this, (byte[])null, 0, -1, -1); } var1 += this.d.b; ByteVector var7 = new ByteVector(var1); var7.putInt(-889275714).putInt(this.b); var7.putShort(this.c).putByteArray(this.d.a, 0, this.d.b); int var8 = 393216 | (this.k & 262144) / 64; var7.putShort(this.k & ~var8).putShort(this.l).putShort(this.n); var7.putShort(this.o); int var9; for(var9 = 0; var9 < this.o; ++var9) { var7.putShort(this.p[var9]); } var7.putShort(var2); for(var3 = this.B; var3 != null; var3 = (FieldWriter)var3.fv) { var3.a(var7); } var7.putShort(var4); for(var5 = this.D; var5 != null; var5 = (MethodWriter)var5.mv) { var5.a(var7); } var7.putShort(var6); if (this.A != null) { var7.putShort(this.newUTF8("BootstrapMethods")); var7.putInt(this.A.b + 2).putShort(this.z); var7.putByteArray(this.A.a, 0, this.A.b); } if (this.m != 0) { var7.putShort(this.newUTF8("Signature")).putInt(2).putShort(this.m); } if (this.q != 0) { var7.putShort(this.newUTF8("SourceFile")).putInt(2).putShort(this.q); } if (this.r != null) { var9 = this.r.b - 2; var7.putShort(this.newUTF8("SourceDebugExtension")).putInt(var9); var7.putByteArray(this.r.a, 2, var9); } if (this.s != 0) { var7.putShort(this.newUTF8("EnclosingMethod")).putInt(4); var7.putShort(this.s).putShort(this.t); } if ((this.k & 131072) != 0) { var7.putShort(this.newUTF8("Deprecated")).putInt(0); } if ((this.k & 4096) != 0 && ((this.b & '\uffff') < 49 || (this.k & 262144) != 0)) { var7.putShort(this.newUTF8("Synthetic")).putInt(0); } if (this.y != null) { var7.putShort(this.newUTF8("InnerClasses")); var7.putInt(this.y.b + 2).putShort(this.x); var7.putByteArray(this.y.a, 0, this.y.b); } if (this.u != null) { var7.putShort(this.newUTF8("RuntimeVisibleAnnotations")); this.u.a(var7); } if (this.v != null) { var7.putShort(this.newUTF8("RuntimeInvisibleAnnotations")); this.v.a(var7); } if (this.w != null) { this.w.a(this, (byte[])null, 0, -1, -1, var7); } if (this.L) { ClassWriter var10 = new ClassWriter(2); (new ClassReader(var7.a)).accept(var10, 4); return var10.toByteArray(); } else { return var7.a; } } } ```

The detailed informantion of the remaining 9 conflicting API pairs can be found in the following attachment. 10 conflicting API pairs in project webmagic-scripts.txt

Dependency tree--

[INFO] us.codecraft:webmagic-scripts:jar:0.7.3 [INFO] +- org.jruby:jruby:jar:1.7.6:compile [INFO] | +- org.jruby:jruby-core:jar:1.7.6:compile [INFO] | | +- org.ow2.asm:asm:jar:4.0:compile [INFO] | | +- org.ow2.asm:asm-commons:jar:4.0:compile [INFO] | | | - org.ow2.asm:asm-tree:jar:4.0:compile [INFO] | | | - (org.ow2.asm:asm:jar:4.0:compile - omitted for duplicate) [INFO] | | +- org.ow2.asm:asm-analysis:jar:4.0:compile [INFO] | | | - (org.ow2.asm:asm-tree:jar:4.0:compile - omitted for duplicate) [INFO] | | +- org.ow2.asm:asm-util:jar:4.0:compile [INFO] | | | - (org.ow2.asm:asm-tree:jar:4.0:compile - omitted for duplicate) [INFO] | | +- org.jruby.joni:joni:jar:2.0.0:compile [INFO] | | | - (org.jruby.jcodings:jcodings:jar:1.0.8:compile - omitted for conflict with 1.0.10) [INFO] | | +- com.github.jnr:jnr-netdb:jar:1.1.2:compile [INFO] | | | - (com.github.jnr:jnr-ffi:jar:1.0.4:compile - omitted for conflict with 1.0.7) [INFO] | | +- com.github.jnr:jnr-enxio:jar:0.4:compile [INFO] | | | +- (com.github.jnr:jnr-constants:jar:0.8.4:compile - omitted for conflict with 0.8.5) [INFO] | | | - (com.github.jnr:jnr-ffi:jar:1.0.3:compile - omitted for conflict with 1.0.4) [INFO] | | +- com.github.jnr:jnr-x86asm:jar:1.0.2:compile [INFO] | | +- com.github.jnr:jnr-unixsocket:jar:0.3:compile [INFO] | | | +- (com.github.jnr:jnr-ffi:jar:1.0.3:compile - omitted for conflict with 1.0.4) [INFO] | | | +- (com.github.jnr:jnr-constants:jar:0.8.4:compile - omitted for duplicate) [INFO] | | | - (com.github.jnr:jnr-enxio:jar:0.4:compile - omitted for duplicate) [INFO] | | +- com.github.jnr:jnr-posix:jar:3.0.1:compile [INFO] | | | +- (com.github.jnr:jnr-ffi:jar:1.0.7:compile - omitted for conflict with 1.0.4) [INFO] | | | - (com.github.jnr:jnr-constants:jar:0.8.5:compile - omitted for conflict with 0.8.4) [INFO] | | +- org.jruby.extras:bytelist:jar:1.0.10:compile [INFO] | | | - (org.jruby.jcodings:jcodings:jar:1.0.5:compile - omitted for conflict with 1.0.8) [INFO] | | +- com.github.jnr:jnr-constants:jar:0.8.5:compile [INFO] | | +- org.jruby.jcodings:jcodings:jar:1.0.10:compile [INFO] | | +- com.github.jnr:jnr-ffi:jar:1.0.7:compile [INFO] | | | +- (com.github.jnr:jffi:jar:1.2.7:compile - omitted for duplicate) [INFO] | | | +- (com.github.jnr:jffi:jar:native:1.2.7:runtime - omitted for duplicate) [INFO] | | | +- (org.ow2.asm:asm:jar:4.0:compile - omitted for duplicate) [INFO] | | | +- (org.ow2.asm:asm-commons:jar:4.0:compile - omitted for duplicate) [INFO] | | | +- (org.ow2.asm:asm-analysis:jar:4.0:compile - omitted for duplicate) [INFO] | | | +- (org.ow2.asm:asm-tree:jar:4.0:compile - omitted for duplicate) [INFO] | | | +- (org.ow2.asm:asm-util:jar:4.0:compile - omitted for duplicate) [INFO] | | | - (com.github.jnr:jnr-x86asm:jar:1.0.2:compile - omitted for duplicate) [INFO] | | +- com.github.jnr:jffi:jar:1.2.7:compile [INFO] | | +- com.github.jnr:jffi:jar:native:1.2.7:compile [INFO] | | +- org.yaml:snakeyaml:jar:1.13:compile [INFO] | | +- com.jcraft:jzlib:jar:1.1.2:compile [INFO] | | +- com.headius:invokebinder:jar:1.2:compile [INFO] | | +- com.martiansoftware:nailgun-server:jar:0.9.1:compile [INFO] | | +- org.jruby:yecht:jar:1.0:compile [INFO] | | - joda-time:joda-time:jar:2.3:compile [INFO] | - org.jruby:jruby-stdlib:jar:1.7.6:compile [INFO] +- org.jetbrains.kotlin:kotlin-stdlib:jar:1.1.2-2:compile [INFO] | - org.jetbrains:annotations:jar:13.0:compile [INFO] +- org.codehaus.groovy:groovy-all:jar:2.1.6:compile [INFO] +- org.python:jython:jar:2.5.3:compile [INFO] +- commons-cli:commons-cli:jar:1.2:compile [INFO] +- junit:junit:jar:4.11:test [INFO] | - org.hamcrest:hamcrest-core:jar:1.3:test [INFO] +- us.codecraft:webmagic-core:jar:0.7.3:compile [INFO] | +- org.apache.httpcomponents:httpclient:jar:4.5.2:compile [INFO] | | +- org.apache.httpcomponents:httpcore:jar:4.4.4:compile [INFO] | | +- commons-logging:commons-logging:jar:1.2:compile [INFO] | | - commons-codec:commons-codec:jar:1.9:compile [INFO] | +- org.apache.commons:commons-lang3:jar:3.1:compile [INFO] | +- us.codecraft:xsoup:jar:0.3.1:compile [INFO] | | +- (org.jsoup:jsoup:jar:1.10.3:compile - version managed from 1.8.3; omitted for duplicate) [INFO] | | - org.assertj:assertj-core:jar:1.5.0:test (scope managed from compile) [INFO] | +- org.slf4j:slf4j-api:jar:1.7.6:compile [INFO] | +- org.slf4j:slf4j-log4j12:jar:1.7.6:compile [INFO] | | +- (org.slf4j:slf4j-api:jar:1.7.6:compile - omitted for duplicate) [INFO] | | - log4j:log4j:jar:1.2.17:compile [INFO] | +- commons-collections:commons-collections:jar:3.2.2:compile [INFO] | +- org.jsoup:jsoup:jar:1.10.3:compile [INFO] | +- commons-io:commons-io:jar:1.3.2:compile [INFO] | +- com.jayway.jsonpath:json-path:jar:2.4.0:compile [INFO] | | +- net.minidev:json-smart:jar:2.3:compile [INFO] | | | - net.minidev:accessors-smart:jar:1.2:compile [INFO] | | | - (org.ow2.asm:asm:jar:5.0.4:compile - omitted for conflict with 4.0) [INFO] | | - (org.slf4j:slf4j-api:jar:1.7.6:compile - version managed from 1.7.25; omitted for duplicate) [INFO] | - com.alibaba:fastjson:jar:1.2.28:compile [INFO] - us.codecraft:webmagic-extension:jar:0.7.3:compile [INFO] +- redis.clients:jedis:jar:2.9.0:compile [INFO] | - org.apache.commons:commons-pool2:jar:2.4.2:compile [INFO] - (us.codecraft:webmagic-core:jar:0.7.3:compile - omitted for duplicate)

code4craft / webmagic

Dependency conflicts on org.ow2.asm:asm, leading to inconsistent program behaviors #951

Dependency tree--

Suggested solutions: