If an OSGI and a non-OSGI plugin are both available, bnd-platform OSGI-ifies the non-OSGI plugin

[nedtwigg]

FANTASTIC PLUGIN - thanks so much!

Gradle doesn't have a good way to consume P2 repositories, which means a lot of people (myself included) are using unpuzzle, a plugin which downloads an Eclipse zip, then puts all of those jars into a local maven repository, which you can then use in Gradle.

This means that some libraries are available twice. e.g. Eclipse has "org.hamcrest.core", which unpuzzle turns into the maven coordinate eclipseMavenGroup:org.hamcrest.core, and Maven Central has that exact same library available as org.hamcrest:hamcrest-core. I've got a big project with tons of dependencies, and it ends up that when I run the bundles task, I end up with a few duplicates:

• org.hamcrest.core_1.3.0.autowrapped.jar from bnd-platform
• org.hamcrest.core_1.3.0.v201303031735.jar from unpuzzle (which is raw from Eclipse)

It's difficult to fix this, because if I need eclipseMavenGroup:something, I automatically end up with eclipseMavenGroup:org.hamcrest.core through transitive dependencies. And if I need org.junit:junit I also end up with org.hamcrest:hamcrest-core through transitives.

Right now, I work around this issue with the following snippet:

platform {
    useBndHashQualifiers = false
}

// for a few plugins, we're getting the OSGI plugin AND a wrapped maven jar
// this deletes the wrapped maven jar, leaving only the OSGI plugin
bundles.doLast {
    def allFiles = project.fileTree('build/plugins').collectNested { it.name }
    def toDelete = []
    allFiles.each { plugin ->
        def split = plugin.indexOf('_') + 1
        def version = plugin.substring(split)
        // if it's a bnd plugin
        if (version.contains('autowrapped')) {
            // get the name (including the _ delimiter)
            def name = plugin.substring(0, split)
            // look for matching non-bnd plugins
            allFiles.each {
                if (it.startsWith(name) && !it.equals(plugin)) {
                    def betterVersion = it.substring(it.indexOf('_') + 1)
                    logger.warn("delete duplicate ${plugin}: ${betterVersion} is better than {version}")
                    project.file('build/plugins/' + plugin).delete()
                }
            }
        }
    }
}

Which outputs this on my project:

delete duplicate com.jcraft.jsch_0.1.51.autowrapped.jar: 0.1.51.v201410302000.jar is better than 0.1.51.autowrapped.jar
delete duplicate org.hamcrest.core_1.3.0.autowrapped.jar: 1.3.0.v201303031735.jar is better than 1.3.0.autowrapped.jar

I think that bnd-platform should realize if it's creating duplicate plugins, and probably throw an error, or at least a warning. It would then be nice if the dsl has a way to say "I know there's a duplicate, I want X", or maybe "everytime there's a duplicate, always drop the wrapped version".

Not urgent though, the workaround above works just fine :)

[stempler]

Great you're finding the plugin useful :smiley: Thanks for the hint on unpuzzle, seems like kind of a counterpart to bnd-platform. I can think of some use cases there this might be handy.

I always thought one of the reasons to use OSGi is that you actually can have multiple versions of a dependency :wink: . Just too bad that in practice this too often results in package uses conflicts and problems with tooling not supporting it properly. But I see how duplicates are a problem if you get the same dependencies through different Maven artifacts as in your case.

What would you classify as duplicate bundles? If symbolic name and major, minor and micro version are the same?

[stempler]

As a side note the issue regarding unpuzzle - what IMHO would also be possible is extending unpuzzle with a map of "known maven artifacts" that allows unpuzzle to recognize them and use the public Maven artifacts instead of the ones downloaded with Eclipse as dependencies (e.g. org.hamcrest:hamcrest-core instead of eclipseMavenGroup:org.hamcrest.core).

[nedtwigg]

extending unpuzzle with a map of "known maven artifacts"

Unpuzzle feels like magic at first because it reads all of the Require-Bundle directives, and encodes them as maven dependencies. But in my experience that mostly turned out to be a trap - the truly OSGi bundles only used Import-Package, which Unpuzzle can't help you with. I think Unpuzzle is just going to be a bridge until Gradle opens up the repository API so that we can build a P2 plugin, but it sure is useful in the meantime!

Since OSGi bundles typically don't encode their maven coordinates, I can't think of a way to implement this besides manually typing in known exclusions. But even if we implemented this for Unpuzzle, I think we'll have similar problems when a real P2 plugin comes around, since it also won't know which of its bundles happens to also be available somewhere on maven.

Your plugin does such a great job mushing real OSGi and maven that I think you're going to get stuck with the hard parts ;-)

What would you classify as duplicate bundles?

When bnd-platform:bundles runs for me, I get this message:

Package com.jcraft.jsch.agentproxy provided by multiple dependencies, using minimal version

It seems that bnd-platform is making some effort to deduplicate packages, so I guess my "least-surprise" expectation would be that native-OSGi and bnd-wrapped would be treated equally, and collapsed down to one version. From the docs, I expected that if I wanted a second version, I would have to specify it manually using auxconf.

Another approach might be to replace the auxconf mechanism with something like this:

resolve('com.jcraft.jsch', '0.1.51.autowrapped', '0.1.51').keep('0.1.51')
resolve('com.google.guava', '17.0', '18.0', '19.0').keepAll()

If there are ever multiple versions available, then you throw an error unless the user correctly anticipated it. This way the user never has multiple versions she didn't expect, and can choose however many of them to keep in the final result.

I just released some of our build infrastructure as poorly documented opensource, and we use this to accomplish something along these lines.

[github-actions[bot]]

This issue has been automatically marked as stale because it has not had activity in the last 60 days. It will be closed in two weeks if no further activity occurs. Thank you for your contributions.

[github-actions[bot]]

This issue has been automatically closed due to inactivity. If it is still valid, please post a comment.