Binary incompatibilities in maven-library used by sbt-native-packager and sbt-pom-reader

[metasim]

This is a recent error, triggered by some combination of SBT version or other plugins. I've been using this plugin as a part of a larger project for a long time, and only had the error triggered when trying to use sbt-native-packager.

It doesn't seem to happen when building the project that uses the plugin independently, but when it serves as a sub-project in a larger muti-project build.

I note that the referenced class is part of maven-aether-provider.

Interested in knowing what additional info I can provide to help identify the issue further.

Backtrace

java.lang.NoClassDefFoundError: org/apache/maven/repository/internal/MavenServiceLocator
    at com.typesafe.sbt.pom.package$.newRepositorySystemImpl(package.scala:23)
    at com.typesafe.sbt.pom.MvnPomResolver$.<init>(MavenPomResolver.scala:20)
    at com.typesafe.sbt.pom.MvnPomResolver$.<clinit>(MavenPomResolver.scala)
    at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:119)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:47)
    at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:18)
    at MavenBuild$.projectDefinitions(build.scala:2)
    at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:496)

SBT Info

[info] This is sbt 0.13.7
[info] The current project is {file:/Users/sfitch/Coding/foo/}bar 2.1-SNAPSHOT
[info] The current project is built against Scala 2.11.4
[info] Available Plugins: sbt.plugins.IvyPlugin, sbt.plugins.JvmPlugin, sbt.plugins.CorePlugin, sbt.plugins.JUnitXmlReportPlugin, sbtmeow.SbtMeowPlugin, diagram.Plugin, com.typesafe.sbt.SbtGit, sbtbuildinfo.Plugin, com.typesafe.sbt.SbtScalariform, net.virtualvoid.sbt.graph.Plugin, com.typesafe.sbt.pom.PomReaderPlugin, com.typesafe.sbteclipse.plugin.EclipsePlugin, org.sbtidea.SbtIdeaPlugin, sbtrelease.ReleasePlugin
[info] sbt, sbt plugins, and build definitions are using Scala 2.10.4

[metasim]

I tried adding the following to my project/plugins.sbt, but still get the error.

libraryDependencies += "org.apache.maven" % "maven-aether-provider" % "3.2.5"

[metasim]

I should have provided the full backtrace, as there's a wrapped exception (ClassNotFoundException):

java.lang.NoClassDefFoundError: org/apache/maven/repository/internal/MavenServiceLocator
    at com.typesafe.sbt.pom.package$.newRepositorySystemImpl(package.scala:23)
    at com.typesafe.sbt.pom.MvnPomResolver$.<init>(MavenPomResolver.scala:20)
    at com.typesafe.sbt.pom.MvnPomResolver$.<clinit>(MavenPomResolver.scala)
    at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:119)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:47)
    at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:18)
    at MavenBuild$.projectDefinitions(build.scala:2)
...
Caused by: java.lang.ClassNotFoundException: org.apache.maven.repository.internal.MavenServiceLocator
    at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
    at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
    at java.security.AccessController.doPrivileged(Native Method)
    at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
    at com.typesafe.sbt.pom.package$.newRepositorySystemImpl(package.scala:23)
    at com.typesafe.sbt.pom.MvnPomResolver$.<init>(MavenPomResolver.scala:20)
    at com.typesafe.sbt.pom.MvnPomResolver$.<clinit>(MavenPomResolver.scala)
    at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:41)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:119)
    ...

[dwalend]

I'm seeing the same problem. No great thoughts beyond your own idea for adding the dependency. If doing that doesn't work, perhaps there's some classpath hierarchy follies in the gears.

[jsuereth]

sbt-native-packager pulls in an incompatible version of maven (because it was a jdeb dependency). I think this should be fixed if you can get one of the newer version (like the 1.0.0-RCs). Not positive. If you're not using JDeb, I think you may be able to exclude the dependency, let me look into that.

[metasim]

Thanks @jsuereth. Back when I had the problem I just ended up converting the sub-project to first-class SBT and forgot about it. Good call with sbt-native-packager. I should have realized the problem with sbt-pom-reader was correlated with my starting to use sbt-native-packager. Excluding JDeb would have probably solved it for me, but it was a good to push me into convert that sub-project :-)

[metasim]

Things seem to be working OK with the 1.0.0 release of sbt-native-packager.

[dwalend]

I found a little time and tried updating to sbt 0.13.8, sbt-pom-reader 2.0.0, with addMavenResolverPlugin . I'm still getting the missing class def for the stack trace.

I'm not using sbt-native-packager. The complexity is all in maven. I'm hoping that doesn't matter much.

What kind of thing should I be looking for?

Thanks,

Dave

java.lang.NoClassDefFoundError: org/apache/maven/repository/internal/MavenServiceLocator
    at com.typesafe.sbt.pom.package$.newRepositorySystemImpl(package.scala:18)
    at com.typesafe.sbt.pom.MvnPomResolver$.<init>(MavenPomResolver.scala:18)
    at com.typesafe.sbt.pom.MvnPomResolver$.<clinit>(MavenPomResolver.scala)
    at com.typesafe.sbt.pom.package$.loadEffectivePom(package.scala:36)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeProjectTree(MavenProjectHelper.scala:121)
    at com.typesafe.sbt.pom.MavenProjectHelper$.makeReactorProject(MavenProjectHelper.scala:48)
    at com.typesafe.sbt.pom.PomBuild$class.projectDefinitions(PomBuild.scala:22)
    at MyBuild$.projectDefinitions(build.scala:3)
    at sbt.Load$.sbt$Load$$projectsFromBuild(Load.scala:496)
    at sbt.Load$$anonfun$27.apply(Load.scala:436)
    at sbt.Load$$anonfun$27.apply(Load.scala:436)
    at scala.collection.immutable.Stream.flatMap(Stream.scala:442)
    at sbt.Load$.loadUnit(Load.scala:436)
    at sbt.Load$$anonfun$18$$anonfun$apply$11.apply(Load.scala:281)
    at sbt.Load$$anonfun$18$$anonfun$apply$11.apply(Load.scala:281)
    at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:91)
    at sbt.BuildLoader$$anonfun$componentLoader$1$$anonfun$apply$4$$anonfun$apply$5$$anonfun$apply$6.apply(BuildLoader.scala:90)
... 

[metasim]

@dwalend Any chance you could dump the output of sbt-dependency-graph of the, um. project-project (?) so we can see what else might be causing the version clash? Now that I think about that, I'm not sure how you do that for the plugins. @jsuereth any tips on dumping the dependency list of the loaded plugins?

[dwalend]

Thanks for picking this up. SHRINE is open source, so you could potentially see everything. It's a sprawling project with some old tech and more than it's share of ugly. I'm trying to use sbt in one subproject before trying everywhere. Try https://open.med.harvard.edu/vvc/viewvc.cgi/shrine/trunk/code/steward/ as starting point if that will help.

[metasim]

@dwalend The plugins.sbt file in that repo path references version 1.0.1. Have you tried it with 2.0.0?

I can give the build a try, but have forgotten how to use svn. Can you provide the checkout command?

[dwalend]

I got sbt-dependency-graph to show me a tree. (I gave up on the ascii graph after about two minutes.) it's got 1456 lines. What's a sane way to share it?

[metasim]

Create a gist? Yeh, don't want the ASCII tree, however. creating a "Dot" version would be helpful.

[dwalend]

gist of the .dot file

Dot is an amazing relic. It's from 1985? Or 1859?

[metasim]

Gotta love the insanity of it!:

[metasim]

@jsuereth Does this look suspicious to you?:

[dwalend]

I'm glad to see it's that small. Someone made SHRINE's svn trunk in early 2008, and that was r611. But all of this looks like sbt's needs, not SHRINE's.

We're looking for out-of-date maven-aether-providers ? (and I just spotted your update.)

[dwalend]

Those M2 versions are coming in from Maven's aether provider. That's pulled in by sbt-maven-resolver, maven-embedder, and sbt-pom-reader. M2 seems kinda sketchy to see.

Have I got the wrong maven aether provider somehow? There's a 3.3.3 version available. However, it seems out-of-model to try to mess with that space. I didn't ask for either version.

What's the right fix? (I do have addMavenResolverPlugin in the plugins.sbt.)

[metasim]

To be honest, at this point, I don't know. Might be worth just trying a different version to see.

[JoshRosen]

Could this issue be addressed by upgrading sbt-pom-reader's own Maven and Aether dependencies? I noticed that this library is using a really old version of Maven. It seems that newer versions are neither source nor binary-compatible with the version used here, so it might be a bit of work to upgrade. Has anyone investigated this?

[metasim]

@JoshRosen What version would you recommend we try?

[metasim]

@JoshRosen @dwalend I have a draft version of the plugin with the Maven/Wagon/Aether dependencies updated. It passes all of its scripted tests, but I'm not convinced all cases are covered, so if you try it out I'd appreciate reports on any bugs you run into.

https://github.com/sbt/sbt-pom-reader/tree/release/2.1.0-RC1 https://bintray.com/sbt/sbt-plugin-releases/sbt-pom-reader/2.1.0-RC1

[metasim]

FYI, RC2 is available (had to back off Maven library version to 3.2.2)

https://github.com/sbt/sbt-pom-reader/tree/release/2.1.0-RC2 https://bintray.com/sbt/sbt-plugin-releases/sbt-pom-reader/2.1.0-RC2

[dwalend]

Got a chance to try RC2 today. (Sorry about taking so long.) Looks like you've fixed this bug, at least for SHRINE (https://open.med.harvard.edu/stash/projects/SHRINE/repos/shrine/browse?at=refs%2Fheads%2Fdevelop It's currently finding 1887 errors, looks like sbt-pom-reader is missing test dependencies and that's already been reported.)

Thanks,

Dave

[jvican]

For an updated version of this, see https://github.com/scalacenter/sbt-pom-reader/commit/f13c8531a77d13d19bb64ae667c9926abae4ca5e and its commit message.