EXCEPTION_ACCESS_VIOLATION when numba and llvmlite are defined as shared modules

[mzdomanska]

• OS Platform, Distribution, and Version: Windows 10
• Python Distribution and Version: 3.9.13
• Java Distribution and Version: OpenJDK 11
• Jep Version: 3.9.1
• Python packages used: numpy (1.22.3), numba (0.56.4), llvmlite (0.39.1)

I'm getting EXCEPTION_ACCESS_VIOLATION when executing the following code

test.py

from numba import jit, int32

@jit(int32(int32, int32), nopython=True)
def f(x, y):
    return x + y

main

public static void main(String[] args) {
    String scriptPath = <path_to_test.py>
    JepConfig jepConfig = new JepConfig()
        .addSharedModules("numpy", "numba", "llvmlite");

    String statement = String.format("f(x=%d, y=%d)", 2, 3);

    try (Interpreter interpreter = new SubInterpreter(jepConfig)) {
      interpreter.runScript(scriptPath);
      interpreter.eval(statement);
    } catch (JepException e) {
      e.printStackTrace();
    }
  }

It works fine with SharedInterpreter, but I have to use SubInterpreter instead because I need virtual environments as well. How can I make it work?

# A fatal error has been detected by the Java Runtime Environment:
#
#  EXCEPTION_ACCESS_VIOLATION (0xc0000005) at pc=0x00007ffa8ec2dcb0, pid=20736, tid=19300
#
# JRE version: OpenJDK Runtime Environment (11.0.6+10) (build 11.0.6+10)
# Java VM: OpenJDK 64-Bit Server VM (11.0.6+10, mixed mode, tiered, compressed oops, g1 gc, windows-amd64)
# Problematic frame:
# C  [python39.dll+0xbdcb0]
#
# No core dump will be written. Minidumps are not enabled by default on client versions of Windows
#
# If you would like to submit a bug report, please visit:
#   http://bugreport.java.com/bugreport/crash.jsp
# The crash happened outside the Java Virtual Machine in native code.
# See problematic frame for where to report the bug.

[bsteffensmeier]

Sometimes these sorts of issues can be resolved if there are other modules that numba and llvmlite are using that can be added to the list of shared modules. Unfortunately shared modules are not an official feature of cpython and may not work with all python modules even with more shared modules. There is nothing we can do to improve compatibility. Sub-Interpreters are an official feature of cpython and any modules that work with cpython sub-interpreters will work with jep sub-interpreters without adding them to shared modules. My recommendation is for you to work with the numba and llvmlite projects to make sub-interpreter compatible versions of their modules.

[mzdomanska]

I see that there's already a feature-request for numba to add suport for subinterpreters - maybe it'll be resolved. I'll also check if there's something that should be added to shared modules. Thanks!