adoptium / adoptium-support

For end-user problems reported with our binary distributions
Apache License 2.0
48 stars 13 forks source link

JDK without JMODs #1198

Closed merks closed 2 days ago

merks commented 2 days ago

Question

The JustJ project is repacking Temurin JDKs using jlink:

https://eclipse.dev/justj/

This has been working well up until I tried it with the JDK 24 beta,

https://ci.eclipse.org/justj/job/build-jres/431/

Which is using JDKs such as this one.

https://github.com/adoptium/temurin24-binaries/releases/download/jdk-24+26-ea-beta/OpenJDK-jdk_x64_windows_hotspot_24_26-ea.zip

Each attempt to use jlink fails like this:

Error: This JDK does not contain packaged modules and cannot be used to create another image with the jdk.jlink module

Indeed the packaging appears to be different:

image

This is the last one with a jmods folder

image

Is this a glitch? JustJ really, really needs JDKs for which jlink works.

Context

Java version:

jdk-24+26-ea-beta

Your operating system and platform:

All these operating systems:

https://ci.eclipse.org/justj/job/build-jres/431/parameters/

HannesWell commented 2 days ago

Can this be related to JEP 493?

karianna commented 2 days ago

See https://github.com/adoptium/temurin-build/issues/4057

merks commented 2 days ago

Thanks for the quick support! Reading the other thread and the JEP, I assume it's expect that jlink should continue to work, just without the actual separate modules in the jmods folder... That would be great because certainly the size of the JDK has been significantly reduced:

image

I.e., sizes that are more similar to the sizes of the full JREs produced by JustJ https://download.eclipse.org/justj/jres/24/downloads/latest/

image

We also produce very small JREs with a subset of modules that are sufficiently functional for the Eclipse Installer (and many other Eclipse RCP-related products) so we really need jlink to continue to work for that purpose:

image

JustJ loves Temurin and https://api.adoptium.net/ ❤️

jerboaa commented 2 days ago

Each attempt to use jlink fails like this:

Error: This JDK does not contain packaged modules and cannot be used to create another image with the jdk.jlink module

@merks The above error suggests you are adding jdk.jlink to the set of modules in the output image (--add-modules jdk.jlink). Does Eclipse justj really need jdk.jlink included? It's one of the restrictions of JEP 493 (that creating a run-time image from a run-time image using jlink doesn't yet allow for jdk.jlink to be included). All other modules should work. If not, that's probably a bug.

The other bug referenced affects Windows/Mac only. Consider experimenting with Linux (which doesn't have the <foo> has been modified issue).

jerboaa commented 2 days ago

Can this be related to JEP 493?

Yes it is. @merks can confirm with --verbose which tells you where it links from.

merks commented 2 days ago

The existing logic simply include all modules. If it's a simple matter of excluding jdk.jlink that would be very good. I can run the script locally for Windows on my Windows machine to try out your suggest and I will do that immediately!

jerboaa commented 2 days ago

The existing logic simply include all modules. If it's a simple matter of excluding jdk.jlink that would be very good.

Great! That matches our experience.

I can run the script locally for Windows on my Windows machine to try out your suggest and I will do that immediately!

FYI: Excluding jdk.jlink on Temurin Windows JDK 24 EA builds will likely fail for you due to (hence suggested to try on Linux if you can): https://github.com/adoptium/temurin-build/issues/4057

merks commented 2 days ago

Unfortunately on Windows I suffer from this problem:

Error: D:\Users\merks\test-jres\jdk-24+26\bin\dt_socket.dll has been modified

I looked closely at the logs and indeed it appears that jlink without jdk.jlink in the module list is working on some of the architectures.

But now that I've filtered the jdk.jlink module from the list, it still fails to build a "most-fully" JRE:

  • jdk-24+26/bin/jlink --verbose --add-modules=java.base,java.compiler,java.datatransfer,java.desktop,java.instrument,java.logging,java.management,java.management.rmi,java.naming,java.net.http,java.prefs,java.rmi,java.scripting,java.se,java.security.jgss,java.security.sasl,java.smartcardio,java.sql,java.sql.rowset,java.transaction.xa,java.xml,java.xml.crypto,jdk.accessibility,jdk.attach,jdk.charsets,jdk.compiler,jdk.crypto.cryptoki,jdk.crypto.ec,jdk.dynalink,jdk.editpad,jdk.graal.compiler,jdk.graal.compiler.management,jdk.hotspot.agent,jdk.httpserver,jdk.internal.ed,jdk.internal.jvmstat,jdk.internal.le,jdk.internal.md,jdk.internal.opt,jdk.internal.vm.ci,jdk.jartool,jdk.javadoc,jdk.jcmd,jdk.jconsole,jdk.jdeps,jdk.jdi,jdk.jdwp.agent,jdk.jfr,jdk.jpackage,jdk.jshell,jdk.jsobject,jdk.jstatd,jdk.localedata,jdk.management,jdk.management.agent,jdk.management.jfr,jdk.naming.dns,jdk.naming.rmi,jdk.net,jdk.nio.mapmode,jdk.sctp,jdk.security.auth,jdk.security.jgss,jdk.unsupported,jdk.unsupported.desktop,jdk.xml.dom,jdk.zipfs --compress=2 --vm=server --output org.eclipse.justj.openjdk.hotspot.jre.full-24-beta-linux-x86_64 Error: This JDK does not contain packaged modules and cannot be used to create another image with the jdk.jlink module

This list of modules should be the full list of modules excluding jdk.incubator and jdk.jlink...

merks commented 2 days ago

BTW, the script I am using is this one:

https://github.com/eclipse-justj/justj/blob/master/releng/org.eclipse.justj.releng/build-jre.sh

It runs on each OS like this for linux x64:

build-jre.sh "https://github.com/adoptium/temurin24-binaries/releases/download/jdk-24+26-ea-beta/OpenJDK-jdk_x64_linux_hotspot_24_26-ea.tar.gz"

So it's pretty easy for anyone to reproduce the problem locally on their machine.

Thanks in advanced for all the help so far!

jerboaa commented 2 days ago

This list of modules should be the full list of modules excluding jdk.incubator and jdk.jlink...

I wonder what the use-case of this "all-modules-jre" is? Wouldn't the default JDK download be an approximation of this for JDK 24? Are jdk.incubator modules an issue?

merks commented 2 days ago

JustJ is well documented so one could read that documentation to understand what it's doing. Right now we have a single shell script that works on all supported operating systems and for JDKs from different providers that can be used to produce JREs with subsets of modules and with different aspects, e.g., with and without debug information/src.zip. Generally this is exactly what JDKs support via jlink. It seems reasonable to expect (or certainly to hope) that it will just continue to work as it has for the past 4 years and as it currently works for Java 11, 17, 21, and 23. In principle it would be possible to create new infrastructure to do it (wrapping as OSGi/p2 installable units) differently for Java 24, but as I mentioned we also produce "full" JREs with debug information/src.zip stripped. I would seem better to get to the bottom of the problem rather than to suggest to avoid using jlink when it doesn't work for reasons unknown. After all, I'm probably not the only one who will expect and hope that jlink generally works for Java 24...

merks commented 2 days ago

Eureka. Excluding the jdk.jpackage module is also necessary:

https://download.java.net/java/early_access/jdk24/docs/api/jdk.jpackage/module-summary.html

Windows and MacOS fail because of signing problems:

Error: C:\Users\genie.justj\jenkins_agent\workspace\build-jres\jdk-24+26\bin\dt_socket.dll has been modified
Error: /Users/genie.justj/jenkins/workspace/build-jres/jdk-24+26/Contents/Home/lib/libjdwp.dylib has been modified 

I think these are both known problem, right? If so, this issue can be closed.

jerboaa commented 2 days ago

Excluding the jdk.jpackage module is also necessary:

Yes, or any module depending on jdk.jlink. When those are necessary, the OpenJDK upstream work-around is to provide jmods for those (linking from JMODs is still supported just fine). For Temurin we plan to not build and package JMODs for JDK 24 for now, unless there is a good reason.

See: https://github.com/adoptium/temurin-build/issues/4035

If you are happy with excluding jdk.jlink and jdk.jpackage for your use cases, then this is good. No need to re-consider shipping JMODs as an extra installable unit.

Windows and MacOS fail because of signing problems:

Error: C:\Users\genie.justj\jenkins_agent\workspace\build-jres\jdk-24+26\bin\dt_socket.dll has been modified
Error: /Users/genie.justj/jenkins/workspace/build-jres/jdk-24+26/Contents/Home/lib/libjdwp.dylib has been modified 

I think these are both known problem, right? If so, this issue can be closed.

Yes, tracked in https://github.com/adoptium/temurin-build/issues/4057

Feel free to re-open if you think there is still something to be done for this.

merks commented 2 days ago

@jerboaa

Thanks for the confirmation and for the help! Once I realized it's probably just a matter of filtering out problematic modules, it didn't take long to stumble upon jdk.jpackage. So I'm very happy with this solution and I appreciate the very quick help from the Adoptium team! 🏆