search

bndtools / bnd

Bnd/Bndtools. Tooling to build OSGi bundles including Eclipse, Maven, and Gradle plugins.
https://bndtools.org
Other
528 stars 306 forks source link

OSGi Java 9 support: Multi-release JAR #2227

Closed njbartlett closed 1 year ago

njbartlett commented 6 years ago

This issue was raised against Felix (maven-bundle-plugin) but it needs to be addressed in bnd: https://issues.apache.org/jira/browse/FELIX-5592

github-actions[bot] commented 2 years ago

This issue has been automatically marked as stale because it has not had recent activity. Given the limited bandwidth of the team, it will be automatically closed if no further activity occurs. If you feel this is something you could contribute, please have a look at our Contributor Guide. Thank you for your contribution.

laeubi commented 2 years ago

Just want to let you know that I started adding MultiRelese support similar to @rotty3000 idea see

but going a step further:

  1. It is now possible to set the desired release on a jar, similar as you can specify --release on the java compiler
  2. depending on the set release, the getResource and alike return a "stacked" unique view on the thing provided as versions, so from the outside it looks like one traditional jar file without multi-release as if I have unpacked and flatten it
  3. It even support versioned manifests and module-infos and one can write out the whole package as a multi-release jar

Based on this, it would be possible to have a "release" flag on the analyzer set, that then analyzes the jar as it would be viewed for the given release (everything else will be hidden, including the versions folder) one can e.g. pass in the BND file, or e.g. like it is done by the maven-compile plugin, put together one multi-release jar (this would then require one invocation for each declared release compiled). This is of course just a first step, but that way one not needs to make everything aware of the multirelease in BND (e.g. a AnalyzerPlugin will not notice anything here).

timothyjward commented 2 years ago

Setting a release flag in the bnd file seems wrong. The normal model is that one bnd file == one bundle, but the release flag would modify this to be one bnd file == one manifest, with multiple bnd files contributing to a single bundle.

If we ignore the layout issues (e.g. how I get stuff into META-INF/versions/X) then from a user perspective it feels like the most usable scenario for getting bnd to generate multi-release manifests would be to set:

Multi-Release: true

in the bnd file. This has the advantage of matching the normal Java header, and being minimal effort for the "happy path" scenario.

The things to clear up would then be stuff like:

laeubi commented 2 years ago

The normal model is that one bnd file == one bundle

I don't think that's really "true" as e.g. BND delegates to bnd instructions of the "parent" already, so there could also be one bnd file == no bundle or even one bnd file = many bundles (if I use the same for many bundles)...

it feels like the most usable scenario for getting bnd to generate multi-release manifests would be to set

But what if I don't want a multi-release jar? I just want a single manifest but for java 11 as a target release. I think this is a conceptual misunderstanding that a user want to build some Multi-Release stuff at all. Still I want BND to discover the same things that java would discover at runtime that is https://docs.oracle.com/javase/9/docs/api/java/util/jar/JarFile.html#getJarEntry-java.lang.String-

So the very first goal (for me) would be that BND could correctly handle Jars like they would be handled in a real executed JVM run, generatinga MR-Bundle is just the second step.

  • This might require additional bnd files.

Now I'm confused, didn't you just said one bnd file == one bundle ;-)

timothyjward commented 2 years ago

The normal model is that one bnd file == one bundle

I don't think that's really "true" as e.g. BND delegates to bnd instructions of the "parent" already, so there could also be one bnd file == no bundle or even one bnd file = many bundles (if I use the same for many bundles)...

This really isn't the case. You may be misunderstanding things if you have only ever used the bnd-maven-plugin which auto-generates you a bnd file based on the pom and other inherited configuration if no bnd file exists. It doesn't matter whether you're using Maven, Gradle, or the bnd workspace model, each project that produces a bundle does so based on a bnd file in that project.

But what if I don't want a multi-release jar? I just want a single manifest but for java 11 as a target release.

If this is what you want then bnd does not need to change because you're not using multi-release. To get this result don't have a META-INF/versions/ folder in your bundle and everything will work as it does now. To get the target release either compile using java 11, or set the execution environment appropriately.

This might require additional bnd files.

Now I'm confused, didn't you just said one bnd file == one bundle ;-)

Seriously? I'm trying to help you here. It is important not to pre-judge the outcome of the discussions about how we do this. I'm open to some pretty significant changes to make this work, including having subsidiary bnd files, but the changes need to make sense and they need to be discussed more widely.

laeubi commented 2 years ago

You may be misunderstanding things if you have only ever used the bnd-maven-plugin which auto-generates you a bnd file based on the pom and other inherited configuration if no bnd file exists.

The bnd-maven-plugin takes a bnd file (or I can embeds one in the XML) and combines them with the "parent" configuration ... so I assume this works similar for other cases as well, but for sure this could be maven-specific.

If this is what you want then bnd does not need to change because you're not using multi-release. To get this result don't have a META-INF/versions/ folder in your bundle and everything will work as it does now.

You mean, it does not work ... whenever I use a MR-bundle as a dependency BND will currently not read it correctly (e.g see https://github.com/bndtools/bnd/issues/5327) and if the dependency contains different packages annotations on the versioned classes variants, or even contains additional classes, BND will not work correctly. Still my bundle will never be a MR jar/bundle and I still want BND to work correctly because if I compile it with target=11 i expect it to work for java 11, weather or not I enable MR or not.

Seriously? I'm trying to help you here.

I'm just pointing out that previously you said multiple BND files "seem bad" and then you say we better should use a alternative approach that might require multiple bnd files :-)

It is important not to pre-judge the outcome of the discussions about how we do this.

At least allow using multiple BND instructions (you are not forced to use them, see previous example where my bundle itself is not MR-Jar) does not limit anything here and you seem to indicate that this is even a speific thing of the bnd-maven-plugin, so why I should it be forbidden to do so, especially when one talks about the bnd-process goal where I'm not building a bundle at all (but generating meta-data)?

I'm open to some pretty significant changes to make this work, including having subsidiary bnd files, but the changes need to make sense and they need to be discussed more widely.

All this does not introduce anything "new" and it is not forbidden right now, so I really curious why this should "not make sense", given that the --release on java is also there, so what from my side makes "no sense" is that BND do not support something similar but want to force me creating a multi-release jar if my only requirement is

  1. BND should at least be able to read dependency jars in a way that is compatible with javac so I don't get different result from BND and the compiler
  2. At least I'm able to generate one manifest for such a release, BND can happily put the burden on me how to actually place this on the right location (either as the default or in a versioned folder or ...)

That's why I think BND needs a -release flag anyways, beside from that it might also include support for specifying a Multi-Release flag that in addition do some magic e.g generate multiple Manifest fragments in one run, check all my classpath to get an idea what versions are all there beside these in the bundle myself (as explained it might has different outcomes!), but this is simply another use-case and I would be happy with having at least the very basic first step.

jonatan-ivanov commented 1 year ago

@pkriens Could you please post a reference to the commit that introduced the fix?

pkriens commented 1 year ago

@jonatan-ivanov the current bnd snapshots contain the handling of MRJs.

pkriens commented 1 year ago

@laeubi I think you are ignoring the all important fact that MRJs mandate that an MRJ must not cause public API differences when run on different VM releases. (This is the main reason why I think MRJs are such a stupid idea; they require this humongous restriction to not fall flat on your face but they have not even the tiniest mechanisms in place to enforce this immense constraint. Ergo, another sun.misc they can fix in Java 42)

The public API exported by the classes in a multi-release JAR file must be exactly the same across versions, ...

No public API changes implies that the only differences that running on different releases can cause are the runtime dependencies. Ergo, bnd should be able to read the public API for any supported version and then the base release is sufficient. This is what we do today.

The spec also says:

A future release of this specification may relax the exact same API constraint to support careful evolution.

I do not think we should try to guess what they're going to do in Java 42 ... Tends to end up badly.

jonatan-ivanov commented 1 year ago

@pkriens

@jonatan-ivanov the current bnd snapshots contain the handling of MRJs.

I'm not sure this answers my question. I did not ask if bnd supports it or not, I asked if you could post a reference to the commit/PR that introduced the fix.

Like this one? https://github.com/bndtools/bnd/issues/5346 Are there more? Can they be linked to this issue?

pkriens commented 1 year ago

This were the primary commits

paulrutter commented 3 months ago

I know this is an old issue, but i read this thread and the felix issue tracker, but cannot find if this is or isn't fixed in the maven bundle plugin. I did find the example that uses the bnd plugin, but is there a Multi-Release instruction for the maven bundle plugin? @pkriens @laeubi

I ran into this when upgrading to Jersey 3.1.7, which is a multi release jar now.

chrisrueger commented 3 months ago

Multi-Release

At least the release notes of bnd version 7.0.0. mention "Supports Multi release JARs" @paulrutter Don't know how this behaves with other plugins you mention in your question.

laeubi commented 3 months ago

You need felix-plugin with latest bnd if felix has not yet updated there is quite a good chance you can simply add bnd-7 to the dependency section of the plugin. Beside that bnd-process goal can often be a replacement for felix-bundle-plugin even though it has some benefits.

paulrutter commented 3 months ago

Thanks, would just adding the dependency be enough or do i need additional instructions? The latest maven bundle plugin release uses bnd 6.3.1, see https://mvnrepository.com/artifact/org.apache.felix/maven-bundle-plugin/5.1.9

laeubi commented 3 months ago

If you don't build a MR jar no additional instructions are required.

paulrutter commented 3 months ago

Thanks, i will try it out and file an issue to the felix issue tracker to get bnd upgraded in the maven bundle plugin if it works.

EDIT

Adding this to the maven-bundle-plugin section indeed solves the build issue with consuming MR jars as compile dependency of the OSGi bundle. 

        <dependencies>
          <!-- Required to upgrade bnd in the maven-bundle-plugin to be able to handle Multi-Release jars -->
          <dependency>
            <groupId>biz.aQute.bnd</groupId>
            <artifactId>biz.aQute.bndlib</artifactId>
            <version>7.0.0</version>
          </dependency>        
        </dependencies>

I will file a PR for the felix project to get it upgraded, although it will be a breaking change due to the java 17 requirement.