IBM Java 8 cannot load a DLL that depends on a newer version of the Microsoft Visual C++ redistributable files than version `14.34.31931.0`.

[lux01]

Java -version output

java version "1.8.0_411"
Java(TM) SE Runtime Environment (build 8.0.8.26 - pwa6480sr8fp26-20240529_01(SR8 FP26))
IBM J9 VM (build 2.9, JRE 1.8.0 Windows Server 2022 amd64-64-Bit Compressed References 20240521_71397 (JIT enabled, AOT enabled)
OpenJ9   - 2a35f43
OMR      - f3321fd
IBM      - a05ee94)
JCL - 20240322_01 based on Oracle jdk8u411-b09

Summary of problem

IBM Java 8 on Windows ships multiple embedded copies of the Microsoft Visual C++ redistributable files in the Visual Studio 2015, 2017, 2019, and 2022 compatibility range. According to https://learn.microsoft.com/en-us/cpp/porting/binary-compat-2015-2017?view=msvc-170, the version of the Microsoft Visual C++ runtime that is made available at runtime must be at least as new as the latest version of the Microsoft Visual Studio compiler used to assemble the entire program.

When a Java application uses the System.loadLibrary() API (or equivalently Runtime.load()/Runtime.loadLibrary()) the entire dependency graph of the loaded DLL falls into scope for what is considered "the entire program" and as such the Visual C++ runtime used to launch the JVM must be at least as high as the newest level used to compile not only the JVM but also any of the DLLs that are loaded by user code.

Due to the way DLL precedence works on windows, the embedded copy of the MSVC runtime will always take preference over any system wide installation (or any other copy on the PATH). This means that standalone java.exe program which loads a DLL that needs a MSVC runtime newer than 14.34.31931.0 will fail to load.

A simple recreate is provided below, build this with the command nmake all using Microsoft Visual Studio 2022 version 17.10 LTSC and observe the following error:

C:\Users\Administrator\Desktop\recreate>java -cp . -Djava.library.path=. Recreate
Exception in thread "main" java.lang.UnsatisfiedLinkError: LibRecreate (A dynamic link library (DLL) initialization routine failed. )
        at java.lang.ClassLoader.loadLibraryWithPath(ClassLoader.java:1464)
        at java.lang.ClassLoader.loadLibraryWithClassLoader(ClassLoader.java:1415)
        at java.lang.System.loadLibrary(System.java:612)
        at Recreate.main(Recreate.java:3)

We do not see this issue when running with IBM Semeru Java 11 or 17, because they both ship new enough embedded versions of the MSVC runtime. Similarly, if we remove the (multiple copies of the) DLLs files for the MSVC runtime from the IBM Java 8 installation then the problem also disappears as the system wide copy takes over.

We encountered this problem when upgrading the version of Visual Studio 2022 that we use to build our product from one minor version to another. We had to do this upgrade to address a number of issues, such as lack of WiX v5 support and this compiler bug.

Diagnostic files

Makefile

all: LibRecreate.dll

clean:
    del /q/s LibRecreate.dll LibRecreate.exp LibRecreate.lib Recreate.cpp.obj LibWithStaticInit.dll LibWithStaticInit.lib LibWithStaticInit.exp LibWithStaticInit.cpp.obj Recreate.h

Recreate.h Recreate.class: Recreate.java
    javac -h . Recreate.java

Recreate.cpp.obj: Recreate.cpp Recreate.h
    cl.exe /std:c++17 /c /Fo:Recreate.cpp.obj Recreate.cpp /EHsc "/I%JAVA_HOME%\include" /MD

LibWithStaticInit.cpp.obj: LibWithStaticInit.cpp LibWithStaticInit.hpp
    cl.exe /std:c++17 /c /Fo:LibWithStaticInit.cpp.obj LibWithStaticInit.cpp /EHsc /MD /DLibWithStaticInit_EXPORTS

LibWithStaticInit.dll LibWithStaticInit.lib: LibWithStaticInit.cpp.obj
    link.exe LibWithStaticInit.cpp.obj /dll /out:LibWithStaticInit.dll

LibRecreate.dll: Recreate.cpp.obj LibWithStaticInit.lib
    link.exe Recreate.cpp.obj LibWithStaticInit.lib /out:LibRecreate.dll /dll 

Recreate.java

public class Recreate {
    public static void main(String[] args) {
        System.loadLibrary("LibRecreate");
        doStuff();
    }

    private static native void doStuff();
}

Recreate.h

/* DO NOT EDIT THIS FILE - it is machine generated */
#include <jni.h>
/* Header for class Recreate */

#ifndef _Included_Recreate
#define _Included_Recreate
#ifdef __cplusplus
extern "C" {
#endif
/*
 * Class:     Recreate
 * Method:    doStuff
 * Signature: ()V
 */
JNIEXPORT void JNICALL Java_Recreate_doStuff
  (JNIEnv *, jclass);

#ifdef __cplusplus
}
#endif
#endif

Recreate.cpp

#include "Recreate.h"

#include "LibWithStaticInit.hpp"

JNIEXPORT void JNICALL Java_Recreate_doStuff(JNIEnv *, jclass) {
    LibWithStaticInit::DoStuff();
}

LibWithStaticInit.hpp

#ifdef LibWithStaticInit_EXPORTS
#define LibWithStaticInit_EXPORT __declspec(dllexport)
#else
#define LibWithStaticInit_EXPORT __declspec(dllimport)
#endif

namespace LibWithStaticInit {
    void LibWithStaticInit_EXPORT DoStuff();
}

LibWithStaticInit.cpp

#include "LibWithStaticInit.hpp"
#include <iostream>
#include <mutex>

namespace {
    std::mutex someMutex;

    int calculateMagicNumber() {
        std::lock_guard<std::mutex> lock(someMutex);
        return 42;
    }

    int magicNumber = calculateMagicNumber();
}

void LibWithStaticInit::DoStuff() {
    std::cout << "DoStuff = " << magicNumber << std::endl;   
}

[lux01]

For completeness, the issue triggered in the above recreate is that the interface used by the C++ STL for std::mutex/std::lock_guard appears to have changed slightly so attempting to lock the mutex results in an attempt to dereference a null pointer in C++.

The native error is

Exception thrown at 0x00007FFCF3EA3008 (msvcp140.dll) in java.exe: 0xC0000005: Access violation reading location 0x0000000000000000.

with native call stack

    msvcp140.dll!mtx_do_lock(_Mtx_internal_imp_t * mtx, const xtime * target) Line 100  C++
    LibWithStaticInit.dll!std::_Mutex_base::lock() Line 52  C++
    LibWithStaticInit.dll!std::lock_guard<std::mutex>::lock_guard<std::mutex>(std::mutex & _Mtx) Line 458   C++
>   LibWithStaticInit.dll!`anonymous namespace'::calculateMagicNumber() Line 9  C++
    LibWithStaticInit.dll!`anonymous namespace'::`dynamic initializer for 'magicNumber''() Line 13  C++
    ucrtbase.dll!_initterm()    Unknown
    LibWithStaticInit.dll!dllmain_crt_process_attach(HINSTANCE__ * const instance, void * const reserved) Line 66   C++
    LibWithStaticInit.dll!dllmain_dispatch(HINSTANCE__ * const instance, const unsigned long reason, void * const reserved) Line 276    C++
    ntdll.dll!LdrpCallInitRoutine() Unknown
    ntdll.dll!LdrpInitializeNode()  Unknown
    ntdll.dll!LdrpInitializeGraphRecurse()  Unknown
    ntdll.dll!LdrpInitializeGraphRecurse()  Unknown
    ntdll.dll!LdrpPrepareModuleForExecution()   Unknown
    ntdll.dll!LdrpLoadDllInternal() Unknown
    ntdll.dll!LdrpLoadDll() Unknown
    ntdll.dll!LdrLoadDll()  Unknown
    KernelBase.dll!LoadLibraryExW() Unknown
    j9prt29.dll!omrsl_open_shared_library(OMRPortLibrary * portLibrary, char * name, unsigned __int64 * descriptor, unsigned __int64 flags) Line 187    C
    j9vm29.dll!classLoaderRegisterLibrary(void * voidVMThread, J9ClassLoader * classLoader, const char * logicalName, char * physicalName, J9NativeLibrary * * libraryPtr, char * errBuf, unsigned __int64 bufLen, unsigned __int64 flags) Line 702 C
    j9vm29.dll!openNativeLibrary(J9JavaVM * vm, J9ClassLoader * classLoader, const char * libName, const char * libraryPath, J9NativeLibrary * * libraryPtr, unsigned __int64(*)(void *, J9ClassLoader *, const char *, char *, J9NativeLibrary * *, char *, unsigned __int64, unsigned __int64) openFunction, void * userData, char * errorBuffer, unsigned __int64 bufferLength) Line 296 C
    j9vm29.dll!registerNativeLibrary(J9VMThread * vmThread, J9ClassLoader * classLoader, const char * libName, const char * libraryPath, J9NativeLibrary * * libraryPtr, char * errorBuffer, unsigned __int64 bufferLength) Line 371    C
    j9vm29.dll!VM_BytecodeInterpreterCompressed::run(struct J9VMThread *)   Unknown
    00000000ffe840a0()  Unknown
    0000000000eef1b8()  Unknown
    jvm.dll!00007ffcf66a77b0()  C
    00000000ffe0e1c8()  Unknown

[pshipton]

This should really go through IBM support. OpenJ9 doesn't control what is included in IBM Java 8 builds.

[lux01]

Thanks @pshipton, Java L3 had advised me to raise it here instead of in Salesforce but I can get one raised if that's more appropriate.

[lux01]

IBM support ticket opened instead