`Duplicate class javax.xml.*` errors when using OkHttp

[christophehenry]

OkHttp, a largely used HTTP client, brings stax:stax-api dependency which also seems to implement the javax.xml.* classes creating a clash with html-unit. Is there a way to solve that?

[rbri]

@christophehenry that might be tricky.... the javax.xml package tree is part of the jdk and as far as i know also part of the android jdk. From my point of view htmlunit does only use something available on the system already - this implies OkHttp might try to add something else.

Do you agree or is there something wrong with my expectations?

[christophehenry]

I'm not sure. All I know that one of Square's projets, Retrofit depends on stax-api which provides javax.xml.* classes and that prevents building:

+--- com.squareup.retrofit2:converter-simplexml:2.9.0
|    +--- com.squareup.retrofit2:retrofit:2.9.0 (*)
|    \--- org.simpleframework:simple-xml:2.7.1
|         +--- stax:stax-api:1.0.1
|         +--- stax:stax:1.2.0
|         |    \--- stax:stax-api:1.0.1

Actually, I might have misread the dependency graph. I don't think it has anything to do OkHttp. It may just be one of Retrofit's converters, namely simplexml that causes the problem. But that may extends to any dependency that provides javax.xml.* classes, possibly any XML marshaller.

[rbri]

Do you use the com.squareup.retrofit2:converter-simplexml in your project - looks like this is optional (https://square.github.io/retrofit/) and maybe you can tell gradle to exclude the whole simple-xml dependency.

For me the stax:stax-api:1.0.1 looks like something from the time wher stax was not part of the java api - the lib is from Aug 31, 2006

[christophehenry]

I don't understand… If it's part of the Android JDK, why does htmlunit-android still embeds these class?

[rbri]

Oh, i think/hope htmlunit-android does not include this class- will check.

[rbri]

ok, now i got your point, will try to find the reason for this....

[rbri]

case is clear for me now - htmlunit-neko uses xerces and xerces uses xml-apis. No real idea at the moment to get rid of this.

[rbri]

ok, have done some work in the background - mainly using my own fork of xerces now. This should solve this problem (and maybe some others).

Will inform via twitter (https://twitter.com/htmlunit) if a new snapshot build is ready for testing.

[rbri]

@christophehenry please try 2.68.0-SNAPSHOT and report your findings here....

Make sure to also have the latest core-js and neko snapshots

[christophehenry]

2.68.0-SNAPSHOT is not present on https://oss.sonatype.org/

[rbri]

@christophehenry now? ;-)

[christophehenry]

I get an class file for org.w3c.dom.traversal.DocumentTraversal not found error with this version.

[rbri]

ok,have updated the snapshot again, please try with the latest build (htmlunit-android-2.68.0-SNAPSHOT.jar - file size 6.785.145 bytes)

[rbri]

Ah my fault, have refreshed again - same file size but hopefully the dependency to DocumentTraversal is gone

[christophehenry]

I refreshed dependecies with ./gradlew build --refresh-dependencies but still having that class file for org.w3c.dom.traversal.DocumentTraversal not found error.

[rbri]

yes i made a mistake - now we have a real new build online (6.773.529 bytes)

[christophehenry]

Hmm… Sorry, that error seems to persist. Even after clearing the cache.

[rbri]

ok, will check again - had a similar case some weeks ago - looks like sometimes gradle does not really update the snapshots.

[rbri]

snapshot again refreshed - file size now 6.168.576 bytes

@christophehenry please try again and (if possible) try to check if you really have the right file

[christophehenry]

No, sorry, same error. But the JAR I have from the Nexus repository is 6.164.721 bytes. So I double checked by manually downloading the one available on the Nexus and comparing the MD5 sum with the one Gradle downloaded. They're the same. So whatever file of 6.168.576 bytes you wished me to test was most likely not published on the Nexus repository.

[rbri]

htmlunit-android-2.68.0-SNAPSHOT_jar.zip

Can you please try this one -> rename it back to jar.

6.123.961 bytes

[rbri]

or you can download the most recent one from https://oss.sonatype.org/content/repositories/snapshots/net/sourceforge/htmlunit/htmlunit-android/2.68.0-SNAPSHOT/ - and again you have to rename....

[christophehenry]

No I can confirm the problem is still very much present in the last version. What's buzzing me, though is the error is very unusual:

cannot access DocumentTraversal
                final HtmlPage page = webClient.getPage("https://htmlunit.sourceforge.io/");
                                               ^
  class file for org.w3c.dom.traversal.DocumentTraversal not found

See, it's pointing on the method accessor operator.

Edit : it is reproductible on a newly created project with the follow snippet:

        new Thread(() -> {
            try (final WebClient webClient = new WebClient()) {
                final HtmlPage page = webClient.getPage("https://htmlunit.sourceforge.io/");
            } catch (IOException e) {
                throw new RuntimeException(e);
            }
        });

[rbri]

really strange....

@christophehenry can you please make a whole test project available (i guess i need an android studio to run it) this will be a great help for me.

[rbri]

@christophehenry looks like i'm able to reproduce your problem - still strange but i can move forward now

[rbri]

OMG!!!! I'm sooooo blind

[rbri]

@christophehenry Can you please try with htmlunit-android 3.0.0-SNAPSHOT and report your results here.

[christophehenry]

Seems to work, now, thx! :thinking:

[rbri]

@christophehenry great and thanks for the quick response.

The version 3 requires some breaking changes for HtmlUnit itself, because of this a real release will not be available soon. Hope this is ok for you.

[rbri]

3.1.0 is there.

Thanks for the report.