The server fails to launch on AIX with Java21 due to a NoClassDefFoundError (NCDFE)

[dazavala]

When starting a server configured with Java21 on AIX, the primordial launch sequence fails immediately with a NoClassDefFoundError indicating class com.ibm.ws.kernel.boot.Launcher,

# wlp/bin/server run

Exception in thread "main" java.lang.NoClassDefFoundError: com.ibm.ws.kernel.boot.Launcher
        at com.ibm.ws.kernel.boot.cmdline.EnvCheck.main(EnvCheck.java:59)
        at com.ibm.ws.kernel.boot.cmdline.EnvCheck.main(EnvCheck.java:37)
Caused by: java.lang.ClassNotFoundException: com.ibm.ws.kernel.boot.Launcher
        at java.base/jdk.internal.loader.BuiltinClassLoader.loadClass(BuiltinClassLoader.java:827)
        at java.base/jdk.internal.loader.ClassLoaders$AppClassLoader.loadClass(ClassLoaders.java:188)
        at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:1102)
        ... 2 more

[dazavala]

The server fails here:

EnvChk
                if (launcher == null) {
59:                 launcher = new Launcher();  // default ctor
                }

BuiltInClassLoader
            protected Class<?> loadClass(String cn, boolean resolve)
                throws ClassNotFoundException
            {
                Class<?> c = loadClassOrNull(cn, resolve);
                if (c == null)
827:                throw new ClassNotFoundException(cn);
                return c;
            }

Initial thoughts:

• The code path and args look correct according to my debugger.

• A java core would give some insight about the command line. Add to jvm.options:

  -Xdump:system+java+snap:events=throw,filter=java/lang/NoClassDefFoundError,request=exclusive+prepwalk+serial+preempt

• Q: Does arg cn="com.ibm.ws.kernel.boot.Launcher" have trailing whitespace. N/A: the wlp install works on Java17

• Q: Is the Class-Path attribute in wlp/bin/tools/ws-server.jar/META-INF/MANIFEST.MF incorrect? Verify jar path in Class-Path: exists in wlp/lib.

  Class-Path: ../../lib/com.ibm.ws.kernel.boot_1.0.94.jar

  N/A: the wlp install works on Java17

• Q: Does the JVM not configure the AppClassLoader (BuiltinClassLoader) URL classpath as expected? Probably, since the wlp install works on Java17

• Requested advice from rt-service (see rt-service slack channel, oct 18)

• If the JVM native code failed to load the class for whatever reason, inquire w/ Java.

[stevenschader]

on the affected AIX with the java21:

contents of the wlp/bin/tools/ws-server.jar/META-INF/MANIFEST.MF

Manifest-Version: 1.0^M
Build-Identifier: SNAPSHOT-Wed Oct 16 03:26:23 UTC 2024^M
Bundle-Copyright: Copyright (c) 1999, 2024 IBM Corporation and others.^M
  All rights reserved. This program and the accompanying materials are^M
  made available under the terms of the Eclipse Public License 2.0 whi^M
 ch accompanies this distribution, and is available at http://www.ecli^M
 pse.org/legal/epl-2.0/.^M
Bundle-License: Eclipse Public License; url=https://www.eclipse.org/le^M
 gal/epl-2.0/^M
Bundle-SCM: connection=scm:git:https://github.com/OpenLiberty/open-lib^M
 erty.git, developerConnection=scm:git:https://github.com:OpenLiberty/^M
 open-liberty.git, url=https://github.com/OpenLiberty/open-liberty/tre^M
 e/release^M
Bundle-SymbolicName: com.ibm.ws.kernel.boot.ws-server^M
Bundle-Vendor: IBM^M
Bundle-Version: 1.0.95.cl241120241016-0303^M
Class-Path: ../../lib/com.ibm.ws.kernel.boot_1.0.95.jar^M
Created-By: 17.0.12 (IBM Corporation)^M
Main-Class: com.ibm.ws.kernel.boot.cmdline.EnvCheck^M
Require-Capability: osgi.ee;filter:="(&(osgi.ee=JavaSE)(version=1.8))"^M
^M
~

-bash-5.2$ ls -l ../../lib/com.ibm.ws.kernel.boot_1.0.95.jar
-rw-r--r--    1 nest     nest         849601 Oct 17 16:11 ../../lib/com.ibm.ws.kernel.boot_1.0.95.jar