Bug Report: Payara doesn't run on latest JDK patch levels / FISH-7665

[lprimak]

Brief Summary

With latest JDK patch levels (20.0.2, 17.0.8, 11.0.20) there is a breaking change that prevents Payara from running at all. I put the following at the end of glassfish/config/asenv.conf to get it to run:

export JAVA_TOOL_OPTIONS
JAVA_TOOL_OPTIONS="-Djdk.util.zip.disableZip64ExtraFieldValidation=true --add-opens=java.base/java.io=ALL-UNNAMED"

Below is the exception:

java.util.zip.ZipException: Invalid CEN header (invalid zip64 extra data field size)
    at java.base/java.util.zip.ZipFile$Source.zerror(ZipFile.java:1750)
    at java.base/java.util.zip.ZipFile$Source.checkExtraFields(ZipFile.java:1276)
    at java.base/java.util.zip.ZipFile$Source.checkAndAddEntry(ZipFile.java:1219)
    at java.base/java.util.zip.ZipFile$Source.initCEN(ZipFile.java:1689)
    at java.base/java.util.zip.ZipFile$Source.<init>(ZipFile.java:1464)
    at java.base/java.util.zip.ZipFile$Source.get(ZipFile.java:1427)
    at java.base/java.util.zip.ZipFile$CleanableResource.<init>(ZipFile.java:723)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:250)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:179)
    at java.base/java.util.zip.ZipFile.<init>(ZipFile.java:193)
    at org.apache.felix.framework.util.SecureAction.openZipFile(SecureAction.java:802)
    at org.apache.felix.framework.util.WeakZipFileFactory$WeakZipFile.<init>(WeakZipFileFactory.java:169)
    at org.apache.felix.framework.util.WeakZipFileFactory$WeakZipFile.<init>(WeakZipFileFactory.java:151)
    at org.apache.felix.framework.util.WeakZipFileFactory.create(WeakZipFileFactory.java:78)
    at org.apache.felix.framework.cache.JarRevision.<init>(JarRevision.java:83)
    at org.apache.felix.framework.cache.BundleArchive.createRevisionFromLocation(BundleArchive.java:804)
    at org.apache.felix.framework.cache.BundleArchive.reviseInternal(BundleArchive.java:491)
    at org.apache.felix.framework.cache.BundleArchive.<init>(BundleArchive.java:226)
    at org.apache.felix.framework.cache.BundleCache.getArchives(BundleCache.java:427)
    at org.apache.felix.framework.Felix.init(Felix.java:798)
    at org.apache.felix.framework.Felix.init(Felix.java:648)
    at com.sun.enterprise.glassfish.bootstrap.osgi.OSGiFrameworkLauncher$1.run(OSGiFrameworkLauncher.java:99)

Expected Outcome

Payara runs

Current Outcome

Payara won't run

Reproducer

$ asadmin start-domain`

Operating System

Any

JDK Version

20.0.2, 17.0.8, 11.0.20

Payara Distribution

Payara Server Full Profile

[Pandrex247]

Thanks Lenny. We'd also noticed it and I've raised an issue on our Jira to look into it - it also affects building the server. As you've already mentioned there are some workarounds using that system property but they're not ideal.

[phillipross]

@lprimak should this workaround work for payara-micro as well? I have arquillian tests using payara-micro via the payara-micro-remote container, but either the jvm args you provided don't work for payara micro or I haven't figured out how to get the maven failsafe plugin to properly propagate them into the container.

[lprimak]

Probably the options aren't getting to Payara Micro. I am not a Micro user so can't help there :( Full profile all the way! I use payara remote container with Arquillian and full profile, works like a charm :)

[phillipross]

woops, i meant to say that i'm using the payara-micro-managed arquillian container (not the remote), but yeah I'll have to dig into some arquillian code and see if I can track down the proper way to propagate all these configs. definitely happy to hear that the workaround works in payara-server-full. thanks again!

[charleech]

Regarding to the Payara Micro Managed Arquillian Container, it mentions as the following: -

cmdOptions Provides additional options to the Java process running the Micro instance (I.e. between java and -jar.).

Then the arquillian.xml for Payara Micro may be the following: -

<?xml version="1.0"?>
<arquillian xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
            xmlns="http://jboss.org/schema/arquillian"
            xsi:schemaLocation="http://jboss.org/schema/arquillian
                http://jboss.org/schema/arquillian/arquillian_1_0.xsd">

    <container qualifier="payara-micro-managed" default="true">
        <configuration>
            <property name="cmdOptions">-Djdk.util.zip.disableZip64ExtraFieldValidation=true --add-opens=java.base/java.io=ALL-UNNAMED</property>
        </configuration>
    </container>
</arquillian>

[phillipross]

cmdOptions Provides additional options to the Java process running the Micro instance (I.e. between java and -jar.).

@charleech The cmdOption property is exactly what I was looking for, thanks! Only the -D system property was needed though. It works for me without the --add-opens parameter.

Also, I don't seem to have any problem with the embedded profile in my tests. Embedded never exhibited the problem, but microprofile did.

[pdurbin]

@Pandrex247 (or anyone) does your following pull request...

• 6348

...which (judging from 8debccc69d408af057d6e8f797f81be1a24960da) seems to have made it into Payara 6.2023.8 (now released) mean that this issue should be closed? Thanks for the fix!

[lprimak]

I’ll need to install the latest Payara and check.

[Pandrex247]

Yes it should now be fixed 🙂

[pdurbin]

@Pandrex247 thanks, yes, I agree, fixed, based on what I'm seeing in my repo: https://github.com/IQSS/dataverse/pull/9795#issuecomment-1683778463

[lprimak]

Well, it's not fixed due to #6362 not being there yet... However, I will close this since it's merged and will be available in the 2023.9 version

[lprimak]

Once #6371 is merged, and released, this should be fully fixed