Add support for running openHAB with Java 21

wborn commented 1 year ago

Today Java 21 will be launched. :rocket: This is a LTS release, so we should also support using it with openHAB.

We can use this issue to keep track of what still needs to be fixed or investigated (similar to #1316).

There's also a launch live stream: https://dev.java/community/java-21-launch/

TODO:

[x] Upgrade Netty to 4.1.99 or newer to prevent JVM crashes (see 4.1.99 release notes) (https://github.com/openhab/openhab-core/pull/3824)
[x] Update Java compatibility check in karaf and karaf.bat

wborn commented 1 year ago

It seems that official Java 21 releases are delayed (see Temurin blog).

Nevertheless I was able to use the Temurin 21-beta+34-202308082331 nightly build and got OH 4.1.0-SNAPSHOT running without issues so far after disabling the recently added Java compatibility check.

It seems the recent Karaf 4.4.4 upgrade helped a lot because OH 4.0.3 does not start with Java 21:

2023-10-02 11:59:11.900 [SEVERE] [org.apache.karaf.main.Main] - Could not launch framework
java.lang.IllegalStateException: Error initializing storage for Equinox container.
    at org.eclipse.osgi.internal.framework.EquinoxContainer.<init>(EquinoxContainer.java:110)
    at org.eclipse.osgi.launch.Equinox.<init>(Equinox.java:53)
    at org.eclipse.osgi.launch.EquinoxFactory.newFramework(EquinoxFactory.java:35)
    at org.eclipse.osgi.launch.EquinoxFactory.newFramework(EquinoxFactory.java:30)
    at org.apache.karaf.main.Main.launch(Main.java:291)
    at org.apache.karaf.main.Main.main(Main.java:183)
Caused by: java.lang.RuntimeException: Error occurred while checking the system module.
    at org.eclipse.osgi.storage.Storage.checkSystemBundle(Storage.java:467)
    at org.eclipse.osgi.storage.Storage.createStorage(Storage.java:187)
    at org.eclipse.osgi.internal.framework.EquinoxContainer.<init>(EquinoxContainer.java:108)
    ... 5 more
Caused by: org.osgi.framework.BundleException: Invalid manifest header Export-Package: "org.osgi.dto;version="1.1",org.osgi.resource;version="1.0",org.osgi.resource.dto;version="1.0";uses:="org.osgi.dto",org.osgi.framework;version="1.10",org.osgi.framework.dto;version="1.10";uses:="org.osgi.dto",org.osgi.framework.hooks.bundle;version="1.1";uses:="org.osgi.framework",org.osgi.framework.hooks.resolver;version="1.0";uses:="org.osgi.framework.wiring",org.osgi.framework.hooks.service;version="1.1";uses:="org.osgi.framework",org.osgi.framework.hooks.weaving;version="1.1";uses:="org.osgi.framework.wiring",org.osgi.framework.launch;version="1.2";uses:="org.osgi.framework",org.osgi.framework.namespace;version="1.1";uses:="org.osgi.resource",org.osgi.framework.startlevel;version="1.0";uses:="org.osgi.framework",org.osgi.framework.startlevel.dto;version="1.0";uses:="org.osgi.dto",org.osgi.framework.wiring;version="1.2";uses:="org.osgi.framework,org.osgi.resource",org.osgi.framework.wiring.dto;version="1.3";uses:="org.osgi.dto,org.osgi.resource.dto",org.osgi.service.condpermadmin;version="1.1.1";uses:="org.osgi.framework,org.osgi.service.permissionadmin",org.osgi.service.packageadmin;version="1.2";uses:="org.osgi.framework",org.osgi.service.permissionadmin;version="1.2",org.osgi.service.resolver;version="1.1";uses:="org.osgi.resource",org.osgi.service.startlevel;version="1.1";uses:="org.osgi.framework",org.osgi.service.url;version="1.0",org.osgi.util.tracker;version="1.5.2";uses:="org.osgi.framework",org.apache.karaf.version;version="4.4.3",org.apache.karaf.jaas.boot.principal;uses:=javax.security.auth;version="4.4.3",org.apache.karaf.jaas.boot;uses:="javax.security.auth,javax.security.auth.callback,javax.security.auth.login,javax.security.auth.spi,org.osgi.framework";version="4.4.3",org.apache.karaf.info;version="4.4.3",, org.apache.karaf.branding, sun.misc, com.sun.jmx.remote.protocol, com.sun.jmx.remote.protocol.jmxmp, org.apache.karaf.jaas.boot.principal;uses:=javax.security.auth;version=4.4.3, org.apache.karaf.jaas.boot;uses:="javax.security.auth,javax.security.auth.callback,javax.security.auth.login,javax.security.auth.spi,org.osgi.framework";version=4.4.3, org.apache.karaf.diagnostic.core;uses:=org.osgi.framework;version=4.4.3, org.apache.karaf.diagnostic.core.common;uses:=org.apache.karaf.diagnostic.core;version=4.4.3"
    at org.eclipse.osgi.util.ManifestElement.parseHeader(ManifestElement.java:355)
    at org.eclipse.osgi.container.builders.OSGiManifestBuilderFactory.createBuilder(OSGiManifestBuilderFactory.java:109)
    at org.eclipse.osgi.storage.Storage.getBuilder(Storage.java:835)
    at org.eclipse.osgi.storage.Storage.checkSystemBundle(Storage.java:415)
    ... 7 more

We might also want to upgrade Netty to 4.1.99.Final or newer because it can crash the JVM, see:

https://netty.io/news/2023/09/21/4-1-99-Final.html

openhab-bot commented 10 months ago

This issue has been mentioned on openHAB Community. There might be relevant details there:

https://community.openhab.org/t/openhab-4-1-release-discussion/152252/44

jlaur commented 9 months ago

@wborn - it seems like the effort to upgrade from Java 17 to 21 is not that big? But I guess this would require a major new version of openHAB, i.e. 5.0 rather than 4.2?

J-N-K commented 9 months ago

"Running with" and "Requiring" are two different things. "Requiring" would indeed be something for a major release, "Running with" is backward compatible and therefore nice to have even for a minor version.

holgerfriedrich commented 9 months ago

To allow compiling and running with Java21 as well, it looks like we need to upgrade a few more things,

[x] ~~ecj to >= 3.36.0 (available)~~
[x] ~~xtend and related libraries for code generation (support libraries seem to break compiling on Java21, and are not released in a version compatible with Java21 yet). This seems a blocker right now.~~
[x] ~~SAT plugin needs to be updated as 0.15.0 does not support spotbugs analysis for Java 21 files~~

I have also seen warnings about zip validation during compilation which could not be suppressed via -Djdk.util.zip.disableZip64ExtraFieldValidation=true in JAVA_TOOL_OPTIONS.

holgerfriedrich commented 8 months ago

@wborn I finally managed to create a distro zip compiled with Java 21 and running on Java 21 🥳 xtend is still a blocker for Java 21, so I had to fix 12 directories (many of the core.model.x folders) to Java 17.

I will open the necessary PRs the next days. A few prerequisites are not yet merged, i.e. the SAT plugin and Karaf 4.4.5 patches. The remaining stuff is basically ecj and a few maven plugins.

holgerfriedrich commented 7 months ago

@kaikreuzer has released SAT 0.16.0 - thanks a lot!

So here comes the first PR for Java 21: openhab/openhab-core#4161.

The question to be decided is how we want to select the target version.

For core, I propose to select automatically based on the JDK release. JDK21 generates Java21 target code (though ~50% of the class files will be Java17 or older, due to embedded dependencies or forced downgrades for xtend), JDK17 generates Java17 code. A profile can be used to manually force JDK21 down to Java17 code.
For addons, I would ~~stick with Java17 unless explicitly selected to avoid problems when dropping plugins into a standard OH release.~~ go with the same approach (changed my mind after I had seem some strange issues during verify). Anyway, there is probably still a lot of work to be done....

WDYT?

And how should we deal with the CI? Shall we add J21 (matrix build)?

holgerfriedrich commented 7 months ago

After some discussion in https://github.com/openhab/openhab-core/pull/4161#discussion_r1545583735, the new proposed approach is as follows:

Allow compilation with JDK 17 and 21, both resulting in Java 17 class files to maintain compatibility with current releases.
Add a profile j21 which can be explicitly activated to generate Java 21 class files.

All PRs have been adapted. @wborn FYI.

holgerfriedrich commented 7 months ago

To update the status here as well: With the PRs mentioned above (https://github.com/openhab/openhab-addons/pull/16612, https://github.com/openhab/openhab-addons/pull/16615, https://github.com/openhab/openhab-core/pull/4171) , everything should be fine:

See my CI builds here: 🥳 https://github.com/holgerfriedrich/openhab-core/actions https://github.com/holgerfriedrich/openhab-addons/action The builds include Java 17, Java 21 compiler to generate Java 17 classes, and Java 21 compiler generating Java 21 classes.

What it still open, is to update the CI to a matrix build. I have not yet opened PRs, but it is there already on private repo above.

wborn commented 6 months ago

Adding Java 21 to the GHA build matrix of the distro and webui repos would be nice as well.

holgerfriedrich commented 6 months ago

Adding Java 21 to the GHA build matrix of the distro and webui repos would be nice as well.

I will add it, thanks for testing the J21 builds.

holgerfriedrich commented 6 months ago

Just for reference: openhab/openhabian#1861 introduced Zulu JDK 21 installation to openHABian

holgerfriedrich commented 6 months ago

@wborn Do you think we are missing something in the scope if this ticket?

OH can now run on Java 21. Maybe issues show up when more people are using is. Marketplace add-ons might run into the same problems with date encoding as we did.

wborn commented 6 months ago

Many thanks for your efforts @holgerfriedrich! I dont know of any other related issues, so let's close this issue and create new issues for any regression or incompatibility that might show up.

openhab / openhab-distro

Add support for running openHAB with Java 21 #1590