Roadmap for Java21 and ending support for 32-bit systems

holgerfriedrich commented 1 month ago

This is a summary of the current state:

Default Java version for openHAB 4.x is Java17. Though, we already support compiling and running openHAB with Java 21. New features of Java versions >17 cannot be used, until we reach the point where we want to end support for Java17. Java17 will reach EOL in Sep 2026. This is the latest possible date we should have completed the transition to Java21 (and as I understand our versioning so far, this should be along with a new major version 5.x of openHAB).

Java21 is typically only available for 64-bit systems, and this ties the decision when to switch to Java21 also to a decision when to give up the 32-bit platform. For us, this is complicating the switch form Java17 to Java21, as we seem to have a big community out there using older Raspberry PIs, RPI3 or even older with limited amount of memory. Now there is at least one Java21 JRE available for 32-bit ARM (and available through openHABian openhab/openhabian#1902). This could allow us to switch to Java21 without/before discontinuation of 32-bit systems.

Discussion of further options:

As we already seen in https://github.com/openhab/openhabian/pull/1902#issuecomment-2391230214 and following, there might be technical reasons for introducing Java21 early before EOL of Java17, and there might also be technical reasons to drop 32-bit systems. I would like to hear the opinion @openhab/maintainers about both Java21 transition and about dropping support for 32-bit systems.

holgerfriedrich commented 1 month ago

My personal opinion:

Java21 could be interesting. Especially the virtual threads could be helpful to simplify our implementation and to speed up code execution. I personally don't want to wait until 2026.
32-bit is still a must for the next year. Probably we have a huge user base still on RPI3 or similar. We should communicate our roadmap early and transparently and recommend the installation of 64-bit images on suitable hardware. PI4 and PI5 are available for quite a while now. I see the concerns by @florian-h05 about supporting 32-bit is preventing us to update to recent libraries. This could force us to do this even before 2026. Maybe @florian-h05 can give more details.

digitaldan commented 1 month ago

Thanks @holgerfriedrich for opening this up. I think having a well defined EOL for 32 bit systems with ample time for users to make the switch sounds like a solid approach. I agree that having to support 32 bit systems in the future is going to slow us down, even with a 32bit JDK21 for PI's. As others have pointed out, many libraries with native bits will have or already have dropped 32 bit builds (like GraalVM). Besides threads, i am also looking forward to pattern matching in switch statements and string literals...... its been a long time coming.

holgerfriedrich commented 1 month ago

Restrictions of dependencies

Apache Derby (used in persistence addons), upgrade to latest version not possible as minimum Java version is Java21. One open CVE. Blocks openhab/openhab-addons#15941, openhab/openhab-addons#15942
GraalJS - latest releases only provided for 64-bit
to be continued

lsiepel commented 1 month ago

I fully support this. When we follow the regular update cycle, there are three options. Summer 2025, Christmas 2025 or summer 2026. If this would signal a mayor update (openHAB 5) it would be nice to bundle it with matter support as (one of the) mayor functional enhancement. Maybe @digitaldan can comment his plans on that addon.

Would a 32bit java21 look like a real solution or would it be crippled? Either way I think it is better to cut the cords and move on as rpi hardware is not that expensive and it’s successor is already available for 5-6 years when this release comes out.

kaikreuzer commented 1 month ago

Christmas 2025 might be a good choice, because people can then put a lot of RasPi 5 under their christmas trees. 🎄 🎄 🎄

wborn commented 1 month ago

A heads-up one year in advance should give everyone plenty of time to prepare for it.

The question I have is will we then also fully drop 32-bit ARM support? That would mean also actively removing 32-bit ARM libraries and Docker images etc.

jlaur commented 1 month ago

The question I have is will we then also fully drop 32-bit ARM support?

If we want to upgrade dependencies that no longer supports 32 bit, I guess this would be needed?

J-N-K commented 1 month ago

The question I have is will we then also fully drop 32-bit ARM support? That would mean also actively removing 32-bit ARM libraries and Docker images etc.

IMO: yes. Something that might not work should either be fixed or removed, otherwise people will start "openHAB works only sometimes". We should avoid that.

florian-h05 commented 1 month ago

@holgerfriedrich Thanks for starting this discussion! I fully agree with the proposed schedule for Christmas 2025 or Summer 2026 👍

The question I have is will we then also fully drop 32-bit ARM support?

IMO yes — even though most parts of openHAB would very likely continue to work if only upgrading the libraries mentioned above, I can already see all the posts „add-on xy not working“ on the forum. And given the increase in resource consumption from Java 11 to Java 17 I think it should again increase from Java 17 to Java 21. This could start causing really bad performance on 32-bit Arm, I would rather have our users buy a new RPi4 2GB for roughly 50€ after they used their old Pi for years, than have unhappy users die to bad performance. And companies like Apple have already killed 32-bit support for their OS years ago … and Windows 11 is 64-bit only (sure, they are mostly x86 not ARM, but anyway).

wborn commented 1 month ago

Fully removing ARM 32-bit support would also make the most sense to me instead of ending up with some untested and degraded user experience. :+1:

lolodomo commented 1 month ago

I am not as enthusiast as you are with the removing of old 32-bit RPI support. Old 32-bit RPI could still be what is the most used by openHAB users (we could start a survey on the community forum to have a more precise proportion of impacted users). This means they will have to buy a new hardware and install & setup everything from scratch. Not something obvious for everybody and time consuming.

Do we know if our big challenger is still supporting 32-bit RPI ? If yes, users may be tempted to keep their old hardware and switch to another solution ?

PS: of course, I will buy a PI 5 myself ;)

florian-h05 commented 1 month ago

Do we know if our big challenger is still supporting 32-bit RPI ? If yes, users may be tempted to keep their old hardware and switch to another solution ?

Their latest Home Assistant OS release still has images for 32-bit Pis (see https://github.com/home-assistant/operating-system/releases/tag/13.1), but their docs only provide links to 64-bit HAOS for Pi 3, 4 and 5 (see https://www.home-assistant.io/installation/raspberrypi#downloading-the-home-assistant-image). From their docs:

We will need a few things to get started with installing Home Assistant.

Raspberry Pi 5 or Raspberry Pi 4 with power supply (Raspberry Pi 3 Model B is ok to get started, but the Model A does not have enough RAM).

So it seems like they still provide HAOS for 32-bit Pis, but they do not mention that in their docs and require you a Pi 4 or Pi 5 (Pi 3 is "okay to get started" only).

I therefore doubt users will keep their old 32-bit Hardware and switch to Home Assistant, where the docs don't even loose a word about their old 32-bit hardware ... and given the HAOS is heavily relying on Docker to provide it's additional services and many Docker images already removed 32-bit ARM support, I think the experience with HAOS on 32-bit ARM isn't that great ...

jlaur commented 1 month ago

I am not so enthusiast as you are with the removing of old 32-bit RPI support. Old 32-bit RPI could still be what is the most used by openHAB

I finally in the beginning of this year upgraded from RPi 3 (32 bit) to 5 (64 bit), and it runs so fast and smooth now - especially startup time and loading/running JavaScript rules. Even though 32 bit is still fully supported, RPi 3 (or even lower) doesn't provide a good user experience anymore IMHO.

This means they will have to buy a new hardware and install & setup everything from scratch.

The more hardware we could support, the better, and I also hate to turn still working hardware into trash. But it's not only about what would be nice - we have to adapt to the world around us as a small part of a bigger ecosystem moving away from 32 bit.

Already in 2018 I had to replace by fit-PC2i with a Fitlet2 for my Linux server, since CentOS had dropped 32 bit support. That was inconvenient for me and kind of a waste, but I fully understand the decision. It's costly, sometimes impossible, trying to keep up with security upgrades etc. when dependencies have moved on.

PS: of course, I will buy a PI 5 myself ;)

You won't regret that. 😄

Fully removing ARM 32-bit support would also make the most sense to me instead of ending up with some untested and degraded user experience

I'm wondering if it would be possible add information to addon.xml whether 32 is supported or not. Perhaps we could then filter and prevent installation of addons no longer supporting 32 bit. This would be kind of a "soft" transition giving users a bit more time to upgrade hardware. The problem would still be users already having such addons installed and then performing an upgrade only to realize these addons will now be broken.

I guess only very few users are still on 32 bit and using Derby (for example). GraalVM is probably a harder case, and maybe it's not worth even considering for this reason?

florian-h05 commented 1 month ago

I guess only very few users are still on 32 bit and using Derby (for example). GraalVM is probably a harder case, and maybe it's not worth even considering for this reason?

For those that don’t want to move away from 32-bit one could compile a JS Scripting add-on version that runs on 32-bit for openHAB 5, it would however won’t be developed anymore …

And as you shared your experience with Pi 3 to Pi 5 upgrade: Its absolutely worth to upgrade, not only to use 64 bit!

holgerfriedrich commented 1 month ago

Openly speaking, I would like to see Java21 way before 2026!

Maybe it is easier to understand for the end user if we introduce both Java21 and end of 32-bit support together in the first OH5 release. But this is keeping us away from using Java21 features for more than a year. Technically, there is at least a solution which allows us to decouple both decisions (but still a risk, only one JRE provider and we don't know how good this works).

Target date Christmas 2025 for Java21 would basically mean that we switch to Java21 in core development mid next year, dropping all Java17 builds and start using Java21 features.

Add-on development would probably still stick to Java17 feature set for at least until the release after. There are still add-ons which are also compiled for older OH releases. I remember the discussions when I started to introduce Java17 features over the whole add-on repo - deliberately done more than 6 months and one release after core had switched to Java17 features.

I would like to hear about your concerns and understand why we should shift this so much towards EOL of Java17....

jlaur commented 1 month ago

Openly speaking, I would like to see Java21 way before 2026!

Me too. However, this:

Christmas 2025 might be a good choice

would be some kind of middle ground (not rushing, not too conservative) allowing us to start developing with Java 21 after 4.4 in the summer next year. If we can decouple the transition to Java 17 from the transition away from 32 bit, we might be able to switch to Java 21 one release faster, i.e. summer 2025 with development cycle 5.0 starting in Christmas this year after releasing 4.3?

One advantage would be that (from my personal observations) maintainers and contributors seem to have more time for openHAB development during wintertime (especially Christmas) than during summertime. Then summer release 2025 would be 5.0.

The only "big thing" I'm aware of right now being in development or planned is the Matter implementation. This would be nice to provide as part of a major new version, so 5.0 won't just be another forced Java upgrade without any big user-oriented improvements, making users really wanting to upgrade. I don't know if this is realistic, @digitaldan?

For users, Christmas is probably a good time for hardware upgrades and reinstallling a major new version of openHAB, but from a development perspective Christmas is kind of a good time for doing the first steps towards a next major release. So this might be somewhat conflicting.

Just some thoughts and wishful thinking... 😄

lsiepel commented 1 month ago

@holgerfriedrich maybe a list of things that have to be done can give some guidance to the timeline. If i understand your post correctly, you would prefer to have a release in summer 2025, meaning OH5 will start development this Christmas. (7 weeks from now). To me that feels somewhat as a rush, buit i don't have full insights to all the things that need to be done. Given that development and review speed is somewhat limited.

I would strongly advice to couple both java21 and removal of 32 bit support into OH5. As these are mayor breaking changes clear communcation is more important then extending the - already subpar - experience on a rpi3 for 6 months. Also don't forget that tehir openHAB will not stop working on the day of OH5 release. Just my 2 cents. And as said before adding matter support to this very same release would be super cool!

mherwege commented 1 month ago

I'm wondering if it would be possible add information to addon.xml whether 32 is supported or not.

While I think this is a good idea, we then should also tackle https://github.com/openhab/openhab-core/issues/4062. Otherwise all marketplace addons will have again to be compiled specifically for an older version, and we get complaints again. If the issue is tackled now for 4.3, we could introduce an extra field in 4.4.

ecdye commented 1 month ago

I would strongly advise to couple both java21 and removal of 32 bit support into OH5. As these are mayor breaking changes clear communcation is more important then extending the - already subpar - experience on a rpi3 for 6 months. Also don't forget that tehir openHAB will not stop working on the day of OH5 release. Just my 2 cents.

I will note that I for one had a openHAB instance up and running in my parents house for upwards of 2 years without ever touching it to upgrade the base openHAB version because of other reasons. It was solid as a rock and I only had to help them remotely restart it maybe twice during that time.

I say this to point out that ultimately if the end user wanted to keep going with their 32 bit machine, they probably could continue right where they were at for a while longer before upgrading.

mike-bike commented 1 month ago

Hi guys, I am not here to comment as a developer but would like to share m point of view as "customer". I think that the shift to 64bit needs to happen sooner or later anyway. From that perspective, I'd appreciate with a major release. It needs to be done, no matter what. Some people may want to stay on their old hardware (even I have a PI2 test system...) but the same situation will be in 2 years. Remember how many Windows 95 and 2k systems are still out there. What I am trying to say is: move on! It is time to change: Matter support and Java 21 should be drivers. If someone wants to keep old hardware they could stay on 4.3. It may sound harsh, but what would be the alternative? Same discussion will be in 2025 or 2026... 32bit is a dying platform! I am myself not keen to migrate without good reason. I do have a 32bit OS on a PI5 to retain compatibility... but so what? It is a good excuse to get another Pi5 and move away from legacy platform. In my PoV the important steps are:

As soon the decision is made: Communicate it clearly and recommend for new users to start with 64bit OS directly to be prepared. Call out 32bit platform as deprecated! End of support will be with 5.0.
Think about RRD - unfortunately it is hardware dependent. Provide scripts to help users making the transition to 64bit now. For the ones who are willing to move, what if backup will be extended to extract RRD DB and load it on a new platform, allowing a seamless as possible transition? What about influxDB or other platform dependent software?

Many thanks for all your great support, effort and enthusiasm to keep this great project alive! I personally consider purchasing another PI5 and move! At the end: it is all about fun!

Kind regards, Michael

lolodomo commented 1 month ago

If today I run Pi OS 64 bit on a PI 5, can I install our current official openHAB distributions without any degradation ?

holgerfriedrich commented 1 month ago

If today I run Pi OS 64 bit on a PI 5, can I install our current official openHAB distributions without any degradation ?

This is what I am running on my development setup. I tend to say "yes". I did not try to transfer my RRD data from 32-bit servers. You might need to dump and restore the RRD data on the new machine, see comment by @mike-bike above.

florian-h05 commented 1 month ago

I totally agree that openHAB 5, 32-bit EOL and Java 21 should be coupled together, even though that means we won‘t be able to use Java 21 features before openHAB 5. IMO it is much more user-friendly to bundle such big changes.

If we plan openHAB 5 for Christmas 2025 (which IMO would be a good schedule), core could start using Java 21 features summer 2025 and users would know that we will drop support for 32-bit one year in advance. And as @ecdye pointed out, you can still continue to run your current openHAB installation for years. Having openHAB 5 in summer 2025 would be fine for me as well, this would allow the use of Java 21 features in core after this Christmas.

Ultimately, we should couple openHAB 5 with Matter support to provide something to the users and not only make it 5.0 for the developers experience. @digitaldan When do you think will the Matter binding ready to be released?

Think about RRD - unfortunately it is hardware dependent. Provide scripts to help users making the transition to 64bit now. For the ones who are willing to move, what if backup will be extended to extract RRD DB and load it on a new platform, allowing a seamless as possible transition?

Didn‘t know RRD is hardware dependent, we should figure out a way to migrate that.

What about influxDB or other platform dependent software?

Unfortunately this is out of scope for us openHAB devs to manage other software. Fortunately however, much of this software communicates to openHAB over the network, so you could move openHAB to a new host and keep this software on the old host. In the case of InfluxDB v2 you can simply backup and restore (see https://docs.influxdata.com/influxdb/v2/backup-restore/). If you are still on v1 because your old host doesn‘t support v2, I would first install v1 on the new host, backup and restore to it, and then upgrade to v2 on the new host (see https://docs.influxdata.com/influxdb/v1/administration/backup_and_restore/ & https://docs.influxdata.com/influxdb/v2/install/upgrade/v1-to-v2/automatic-upgrade/).

digitaldan commented 1 month ago

When do you think will the Matter binding ready to be released?

The binding is not in bad shape now and supports a lot of device types. I'm living with it in my production system with about 20 devices and am pretty happy. The only thing stopping me from doing an official beta is i want to make sure i lock down the general thing and channel structure from changing , right now i have the freedom to make sweeping changes and not affect too many people.

I also am thinking about breaking out the runtime as its own addon (something like org.openhab.io.matter-runtime) as i plan next to create a org.openhab.io.matter-bridge addon which will expose openHAB items as matter devices for local control by 3rd party systems (this is similar to Homekit or the Hue Emulation binding). So i want to lock that dependency structure down before a beta.

This is a long way to say, December 2024 might be tough for an official release, but i will definitely have a beta, possibly in the marketplace for users to test with. But summer 2025 would likely be when openHAB officially ships with it.

florian-h05 commented 1 month ago

Great to hear your progress!

But summer 2025 would likely be when openHAB officially ships with it.

Then that’s a good reason to release openHAB 5 in summer I think. Someone above also said that it seems our devs have more time during winter, so from dev perspective it would make sense as well.

lsiepel commented 1 month ago

+1 for summer release.

Slightly off-topic: are we ready to remove the band-aid for other parts like DSL Rules, or are there other parts that we should 'clean-up' ?

florian-h05 commented 1 month ago

IIRC @J-N-K was thinking about refactoring rules DSL to an add-on for openHAB 4. I don’t know the background behind this idea, maybe Jan can tell us more about it.

florian-h05 commented 3 weeks ago

@kaikreuzer How should we make an „official“ decision here so we can announce it when releasing 4.3?

kaikreuzer commented 2 weeks ago

The discussion found its way to have openHAB 5 next summer. I would say, if nobody from @openhab/maintainers vetos against this in the coming days (let's say by next weekend), we can consider this a joint decision.

holgerfriedrich commented 1 week ago

@mike-bike

2. Think about RRD - unfortunately it is hardware dependent. Provide scripts to help users making the transition to 64bit now. For the ones who are willing to move, what if backup will be extended to extract RRD DB and load it on a new platform, allowing a seamless as possible transition? What about influxDB or other platform dependent software?

FYI: RRD is hardware dependent, RRD4J is not. We use RRD4J, so we don't run into an issue here. See https://github.com/openhab/openhab-docs/issues/2395#issuecomment-2474703885.

dilyanpalauzov commented 1 week ago

I wanted to switch from RPI3b to RPI4, both 32bit but failed. The reason is that the WLAN module in RPI4 is worse than in RPI3. I compiled the newset version of wpa_supplicant, switched from 60 to 50 Hz, reduced the resolution as there is interference between WLAN and HDMI. No matter what I tried, and I tried a lot, the WLAN module of RPI4 is not as reliable as in RPI3. In RPI4, when it is far away from the router, sometimes it connects, sometimes it does not, while RPI3 always connects from the same place. So at the end I returned to RPI3b.

Why running 64bit Java on RPI3 is not an option, does it need too much RAM? In any case, to speed up the whole thing I moved from JS Scripting to Groovy, as I think the latter requires less RAM (has smaller jar file anyway).

For reducing the RAM needed by OH there is a different ticket, but I think OpenAPI, Language Server, voice, model.ide.* (or alike) should be converted to add ons. These might be initially installed, but the user should have the possibility to uninstall them from the Market place (not to stop them in Karaf -utterly complicated to find out what can I stop, what I do not need, without breaking what I need, after upgrade Karaf forgets which bundles were stopped). Ultimately a scripting language, which has very low runtime memory overhead (XTend, but not in it current form as DSL; or Java as JSR223 langauge) can keep OH fully functinal on low memory 64bit devices. So if less RAM is required by OH in 64bit mode, it can still run on RPI3. If all addons I use (I do not user GraalVM) continue running on 32bit, I will stay on 32bit.

That said, I see no problem if some addons run only on 64bit and some on both 32 and 64 bit, as long as this is documented.

How is Matter support related to Java 21 or 64 bit? Is the intended release date by coincidence just the same, or there are more dependencies?

lolodomo commented 1 week ago

How is Matter support related to Java 21 or 64 bit? Is the intended release date by coincidence just the same, or there are more dependencies?

Matter currently works with Java 17 32 bit so this is fully unrelated.

openhab / openhab-distro