bnd dependencies are not reflected as maven dependencies

[bosschaert]

The Bnd build has its own dependency mechanism. For the compilation stage these dependencies are dynamically added to the maven classpath, but they are not visible directly in the pom.xml. This may be an issue with other tools that use the maven dependencies in other phases to achieve some task.

This can most easily be observed by creating a simple bndtools project that depends on something, e.g. osgi.core and implementing an activator. To see it run the following command on bnd-maven-plugin-parent/samples/sample-projects/TestBundle:

$ mvn dependency:tree
[INFO] Scanning for projects...
[INFO] + /Users/David/clones/bnd/bnd-maven-plugin-parent/samples/sample-projects/TestBundle
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building TestBundle 1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ TestBundle ---
[INFO] org.coderthoughts.demo.bndmvn:TestBundle:bundle:1.0.0-SNAPSHOT
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------

It would be nice if the bndtools dependencies could be represented in the pom.xml as well, possibly by generating the <dependencies> section, which would make integration with other maven plugins better.

[pkriens]

David, are you planning to fix this for 2.4?

[theHilikus]

So the idea would be to really control them in the bnd file and have them generated in the pom? isn't this some kind of catch-22 where you would need to run maven to run the bnd-maven-plugin to generate the dependencies that maven needs? In other words, if the bnd-maven-plugin is attached to the package phase, and it's only there where the pom gets its dependencies populated, it is probably already too late, most phases that would use the dependency information already passed. I'm not even sure maven will like a modification to the pom in the middle of a run

Am I missing something?

--Alejandro

[bosschaert]

So I have been thinking a little bit about this...

Basically the bnd file should be the master, but the pom.xml could contain them as dependencies.

A couple of thoughts around this:

• if the artifact referenced by bnd was originally released by Maven, this isn't too hard, as the META-INF/maven location inside the artifact contains its coordinates, which bndtools could read out and put in the pom.xml.
• for artifacts that don't come from Maven, we might be able to use system dependencies and point at the files in the cnf directory.

I think the maintenance of the pom.xml file is actually not done by bnd or the bnd-maven-plugin but should really be done by the bndtools IDE. Therefore this issue should move to bndtools/bndtools as issue.

[bosschaert]

I created an issue for it in bndtools (https://github.com/bndtools/bndtools/issues/851). Unfortunately github doesn't support moving issues :(

[theHilikus]

ok, so as soon as you add a dependency in the bnd file, bndtools will modify the pom.xml, which you will need to commit so that it can be used in a CI server. is that it? that makes sense to me. no catch-22.

On second thought there are some issues that might be problematic: 1) if the dependencies in the bnd file are less configurable than the maven dependencies. Example, you can declare a maven dependency as a runtime-only dependency, or a test-only dependency. So bndtools would need to have a way to be as expressive as maven 2) i don't know if you can define non-code dependencies in bndtools. In maven i often have dependencies of type pdf, which I use in the maven-assembly-plugin to construct a kind of deployable artifact (think a subsystem)

[bosschaert]

ok, so as soon as you add a dependency in the bnd file, bndtools will modify the pom.xml

Yep. Correct.

On your additional thoughts, I would suggest we get the basics working first and then look at the edge cases.

[bjhargrave]

Keeping the POM in sync with the bnd files seems a recipe for disaster. If the bnd-maven-plugin can handle the dependencies for compiler, why not for all phases?

Also, bnd can be used without Eclipse and bndtools, so a solution can't just be bndtools based.

[pkriens]

I don't see any practical problems in this bug reports. Can you give some actual concrete use cases that fail? And are these worth this potential quagmire? We always knew that there are limitations, the scope of this plugin is to build bnd with maven, not to be be maven, there are other tools for that and we all know there are serious limitations around that model.

I am perfect ok with limitations in this model because we will never be able to satisfy 100% maven fidelity.

[bosschaert]

One practical problem is shown in the initial issue text above, i.e. that the mvn dependency:tree command doesn't display the correct information: the bnd dependencies are not shown. Another real practical problem is that this setup won't work with the Maven Assembly Plugin, which is a very common tool to be used with Maven. Bndtools cannot do this for Maven, because it doesn't even get involved by Maven when the Assembly plugin is invoked.

[pkriens]

So this plugin does not allow the bnd plugin to do its magic? That looks like an error in the plugin doesn't it? It should allow other plugins to setup the classpath, that is inherent in the maven model? Don't others then have the same problem?

[bosschaert]

If I understand it well, the assembly plugin (and other maven plugins) purely work on the dependencies in the pom. AFAIK there is no plugin model to allow other entities (such as bnd) to provide these dependencies.

[pkriens]

Did you try out a POM that has the dependencies listed, which are I guess system and contain a file path, and see how the tree and assembly plugin work in that case? I expect that the utility would not be very high in many cases (in the tree command it would only provide one level since we have not transitivity). What would the assembly plugin do?

I think we're stuck between a rock and a hard place here (well, our potential users). Rather spent effort on improving the GUI and other tools than trying to outcompete maven on its own turf since that is a loosing game by definition.

[bosschaert]

Did you try out a POM that has the dependencies listed, which are I guess system and contain a file path, and see how the tree and assembly plugin work in that case?

I haven't tried it yet, but I will soon :)

[timothyjward]

So this plugin does not allow the bnd plugin to do its magic? That looks like an error in the plugin doesn't it? It should allow other plugins to setup the classpath, that is inherent in the maven model? Don't others then have the same problem?

@bnd Other plugins that won't work are the eba-maven-plugin and the maven-compiler-plugin. All of these plugins use the standard maven API for working out which other projects this project depends on. There is no error in these plugins - they are using the correct Maven APIs to work our dependencies. The problem is that in maven these dependencies always come from the pom. In the bnd model there are no dependencies in the pom, hence this bug.

There is also the problem that bnd won't always be run as part of the build. If I build another maven project that depends on a bundle built with the bnd-maven-plugin then I won't see any of its dependencies unless they are in the pom. At the very least we need to write the dependencies into the pom that gets packaged into the bundle. That way when someone depends on a bundle built with the bnd-maven-plugin it will work properly in the maven world.

[fhuberts]

I tried to make this clear during the gradle fight but you guys didn't listen. And now you're stuck with the same proiblem here in maven as you have created in the 'much improved' (not) gradle build.

It's a huge mistake to let bnd drive the build tool, it must be the other way around. You don't let the compiler drive make, now do you?

[bjhargrave]

That is a false analogy. In gradle, as in ant, bnd supplied information is used to establish the java dependencies (e.g. classpath for compilation). All other parts of the gradle build can see and use that information.

It would appear here that the bnd-maven-plugin is not designed to or, by maven's design, is not able to always establish the java dependencies such that all other parts of the maven build can see them. Perhaps the bnd-maven-plugin design needs to be looked at. There may be a better way for it to establish these dependencies at a earlier stage.

[fhuberts]

Ok then, for the last time:

Your gradle plugin assumes that bnd contains all information that is needed to build everything in the build. This is the mistake: you can't assume that. bnd is the compiler, not the build tool.

Therefore the analogy is valid.

[bjhargrave]

No. The gradle plugin assumes bnd contains all the information needed to build the bundle in the project being built including the java dependencies which is a valid assumption given the design of bnd and the bnd file. bnd is not the build tool. But it does describe the dependencies for the bundle project which the gradle plugin uses to parameterize the compiler. The gradle plugin then uses bnd to assemble the bundle.

Each project's build is independent. There can be non-bundle projects in the overall build and those projects will control their own building.

[timothyjward]

Ideally we need the bnd-maven-plugin to do something similar to what the gradle plugin does, and inject the dependencies from the bnd file into the maven model (including maven coordinates if we can find them). I'm not aware of a primordial lifecycle step for this in maven, which may mean that it can't be done.

If people want to use maven to control bnd instead of the bnd.bnd file then that's the job of the maven-bundle-plugin, which we're not trying to replace.

[pkriens]

@Tim: the plugin takes the bnd buildpath and adds them to the resolved classpath. Just talked to David who will do a quick test to see if we can add them the project artifacts, which I assume will then be resolved later by maven into a classpath

[bosschaert]

Just tried Peter's experiment which was the following change in ConfigureMavenProject:

-                       project.setResolvedArtifacts(classpath);
+                       project.getArtifacts().addAll(classpath);

But unfortunately it didn't work. In fact the build-time dependencies are not working with that either (they do work with the setResolvedArtifacts API). So we need to experiment more.

[timothyjward]

Other people have tried this sort of thing before, but without too much success:

http://stackoverflow.com/questions/16070149/how-can-i-programmatically-add-an-artifact-to-the-mavenproject

We should also be careful when using setResolvedArtifacts() - it's not actually API

http://maven.apache.org/ref/3.0/maven-core/apidocs/org/apache/maven/project/MavenProject.html#setResolvedArtifacts(java.util.Set)

[bjhargrave]

Closing. See #629.