search

LeeKamentsky / python-javabridge

Python wrapper for the Java Native Interface
Other
116 stars 63 forks source link

Failed to find class org/cellprofiler/runnablequeue/RunnableQueue #195

Open neilyio opened 3 years ago

neilyio commented 3 years ago

I'm having trouble running javabridge on my MacOS Big Sur (M1) with Azul Zulu OpenJDK. pip install works correctly, though I have to set CFLAGS to make the numpy dependency compile in my virtualenv:

export CFLAGS="-I${VENV_PATH}/lib/python3.9/site-packages/numpy/core/include $CFLAGS"

I'm running:

javabridge.start_vm(
    run_headless=True, class_path=classpath_args
)

where classpath_args is:

['src',
 'resources',
 '/Users/neil/.m2/repository/cider/cider-nrepl/0.25.5/cider-nrepl-0.25.5.jar',
 '/Users/neil/.m2/repository/clj-python/libpython-clj/2.000/libpython-clj-2.000.jar',
 '/Users/neil/.m2/repository/funcool/cuerdas/2021.05.29-0/cuerdas-2021.05.29-0.jar',
 '/Users/neil/.m2/repository/http-kit/http-kit/2.5.3/http-kit-2.5.3.jar',
 '/Users/neil/.m2/repository/nrepl/nrepl/0.8.3/nrepl-0.8.3.jar',
 '/Users/neil/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar',
 '/Users/neil/.m2/repository/org/clojure/clojurescript/1.10.879/clojurescript-1.10.879.jar',
 '/Users/neil/.m2/repository/org/clojure/core.async/1.3.618/core.async-1.3.618.jar',
 '/Users/neil/.m2/repository/org/clojure/data.json/2.4.0/data.json-2.4.0.jar',
 '/Users/neil/.m2/repository/refactor-nrepl/refactor-nrepl/2.5.1/refactor-nrepl-2.5.1.jar',
 '/Users/neil/.m2/repository/cnuernber/dtype-next/8.024/dtype-next-8.024.jar',
 '/Users/neil/.m2/repository/net/java/dev/jna/jna/5.7.0/jna-5.7.0.jar',
 '/Users/neil/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar',
 '/Users/neil/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar',
 '/Users/neil/.m2/repository/com/cognitect/transit-clj/0.8.309/transit-clj-0.8.309.jar',
 '/Users/neil/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20210505/closure-compiler-unshaded-v20210505.jar',
 '/Users/neil/.m2/repository/org/clojure/google-closure-library/0.0-20201211-3e6c510d/google-closure-library-0.0-20201211-3e6c510d.jar',
 '/Users/neil/.m2/repository/org/clojure/tools.reader/1.3.3/tools.reader-1.3.3.jar',
 '/Users/neil/.m2/repository/org/clojure/tools.analyzer.jvm/1.1.0/tools.analyzer.jvm-1.1.0.jar',
 '/Users/neil/.m2/repository/camel-snake-kebab/camel-snake-kebab/0.4.2/camel-snake-kebab-0.4.2.jar',
 '/Users/neil/.m2/repository/com/github/wendykierp/JTransforms/3.1/JTransforms-3.1.jar',
 '/Users/neil/.m2/repository/insn/insn/0.5.2/insn-0.5.2.jar',
 '/Users/neil/.m2/repository/it/unimi/dsi/fastutil/8.2.1/fastutil-8.2.1.jar',
 '/Users/neil/.m2/repository/org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.jar',
 '/Users/neil/.m2/repository/org/ow2/asm/asm/9.0/asm-9.0.jar',
 '/Users/neil/.m2/repository/org/roaringbitmap/RoaringBitmap/0.9.0/RoaringBitmap-0.9.0.jar',
 '/Users/neil/.m2/repository/org/xerial/larray/larray-mmap/0.4.1/larray-mmap-0.4.1.jar',
 '/Users/neil/.m2/repository/primitive-math/primitive-math/0.1.6/primitive-math-0.1.6.jar',
 '/Users/neil/.m2/repository/techascent/tech.resource/5.04/tech.resource-5.04.jar',
 '/Users/neil/.m2/repository/com/cognitect/transit-java/0.8.332/transit-java-0.8.332.jar',
 '/Users/neil/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20201211-3e6c510d/google-closure-library-third-party-0.0-20201211-3e6c510d.jar',
 '/Users/neil/.m2/repository/org/clojure/core.memoize/1.0.236/core.memoize-1.0.236.jar',
 '/Users/neil/.m2/repository/org/clojure/tools.analyzer/1.0.0/tools.analyzer-1.0.0.jar',
 '/Users/neil/.m2/repository/pl/edu/icm/JLargeArrays/1.5/JLargeArrays-1.5.jar',
 '/Users/neil/.m2/repository/org/roaringbitmap/shims/0.9.0/shims-0.9.0.jar',
 '/Users/neil/.m2/repository/org/xerial/larray/larray-buffer/0.4.1/larray-buffer-0.4.1.jar',
 '/Users/neil/.m2/repository/org/clojure/tools.logging/1.1.0/tools.logging-1.1.0.jar',
 '/Users/neil/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.8.7/jackson-core-2.8.7.jar',
 '/Users/neil/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar',
 '/Users/neil/.m2/repository/org/msgpack/msgpack/0.6.12/msgpack-0.6.12.jar',
 '/Users/neil/.m2/repository/org/clojure/core.cache/1.0.207/core.cache-1.0.207.jar',
 '/Users/neil/.m2/repository/com/googlecode/json-simple/json-simple/1.1.1/json-simple-1.1.1.jar',
 '/Users/neil/.m2/repository/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar',
 '/Users/neil/.m2/repository/org/clojure/data.priority-map/1.0.0/data.priority-map-1.0.0.jar']

And I'm getting the following output.

Failed to find class org/cellprofiler/runnablequeue/RunnableQueue

Traceback (most recent call last):
  File "/Users/neil/Library/Caches/pypoetry/virtualenvs/blisp-97-MdyRi-py3.9/lib/python3.9/site-packages/javabridge/jutil.py", line 327, in start_thread
    vm.create_mac(args, RQCLS, library_path, libjli_path)
  File "_javabridge.pyx", line 709, in _javabridge.JB_VM.create_mac
RuntimeError: Failed to create Java VM. Return code = -1
Failed to create Java VM

A couple print() statements in javabridge/jutil.py showed me that the start_thread() function is finding library_path and libjli_path, each at:

/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/server/libjvm.dylib
/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib

start_thread() is failing at the next line after that, which is vm.create_mac(args, RQCLS, library_path, libjli_path).

Hoping it's something simple I'm doing wrong!

LeeKamentsky commented 3 years ago

Hi Neil, Sorry for your difficulties. I don't have a Mac, so I can't recreate this error. Are /Library/Java/JavaVirtualMachines/zulu-11. jdk/Contents/Home/lib/server/libjvm.dylib and /Library/Java/ JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib present on your computer?

On Mon, Oct 11, 2021 at 11:12 AM Neil Hansen @.***> wrote:

I'm having trouble running javabridge on my MacOS Big Sur (M1) with Azul Zulu OpenJDK. pip install works correctly, though I have to set CFLAGS before running:

I'm running:

javabridge.start_vm( run_headless=True, class_path=classpath_args )

where classpath_args is:

['src', 'resources', '/Users/neil/.m2/repository/cider/cider-nrepl/0.25.5/cider-nrepl-0.25.5.jar', '/Users/neil/.m2/repository/clj-python/libpython-clj/2.000/libpython-clj-2.000.jar', '/Users/neil/.m2/repository/funcool/cuerdas/2021.05.29-0/cuerdas-2021.05.29-0.jar', '/Users/neil/.m2/repository/http-kit/http-kit/2.5.3/http-kit-2.5.3.jar', '/Users/neil/.m2/repository/nrepl/nrepl/0.8.3/nrepl-0.8.3.jar', '/Users/neil/.m2/repository/org/clojure/clojure/1.10.1/clojure-1.10.1.jar', '/Users/neil/.m2/repository/org/clojure/clojurescript/1.10.879/clojurescript-1.10.879.jar', '/Users/neil/.m2/repository/org/clojure/core.async/1.3.618/core.async-1.3.618.jar', '/Users/neil/.m2/repository/org/clojure/data.json/2.4.0/data.json-2.4.0.jar', '/Users/neil/.m2/repository/refactor-nrepl/refactor-nrepl/2.5.1/refactor-nrepl-2.5.1.jar', '/Users/neil/.m2/repository/cnuernber/dtype-next/8.024/dtype-next-8.024.jar', '/Users/neil/.m2/repository/net/java/dev/jna/jna/5.7.0/jna-5.7.0.jar', '/Users/neil/.m2/repository/org/clojure/core.specs.alpha/0.2.44/core.specs.alpha-0.2.44.jar', '/Users/neil/.m2/repository/org/clojure/spec.alpha/0.2.176/spec.alpha-0.2.176.jar', '/Users/neil/.m2/repository/com/cognitect/transit-clj/0.8.309/transit-clj-0.8.309.jar', '/Users/neil/.m2/repository/com/google/javascript/closure-compiler-unshaded/v20210505/closure-compiler-unshaded-v20210505.jar', '/Users/neil/.m2/repository/org/clojure/google-closure-library/0.0-20201211-3e6c510d/google-closure-library-0.0-20201211-3e6c510d.jar', '/Users/neil/.m2/repository/org/clojure/tools.reader/1.3.3/tools.reader-1.3.3.jar', '/Users/neil/.m2/repository/org/clojure/tools.analyzer.jvm/1.1.0/tools.analyzer.jvm-1.1.0.jar', '/Users/neil/.m2/repository/camel-snake-kebab/camel-snake-kebab/0.4.2/camel-snake-kebab-0.4.2.jar', '/Users/neil/.m2/repository/com/github/wendykierp/JTransforms/3.1/JTransforms-3.1.jar', '/Users/neil/.m2/repository/insn/insn/0.5.2/insn-0.5.2.jar', '/Users/neil/.m2/repository/it/unimi/dsi/fastutil/8.2.1/fastutil-8.2.1.jar', '/Users/neil/.m2/repository/org/apache/commons/commons-math3/3.6.1/commons-math3-3.6.1.jar', '/Users/neil/.m2/repository/org/ow2/asm/asm/9.0/asm-9.0.jar', '/Users/neil/.m2/repository/org/roaringbitmap/RoaringBitmap/0.9.0/RoaringBitmap-0.9.0.jar', '/Users/neil/.m2/repository/org/xerial/larray/larray-mmap/0.4.1/larray-mmap-0.4.1.jar', '/Users/neil/.m2/repository/primitive-math/primitive-math/0.1.6/primitive-math-0.1.6.jar', '/Users/neil/.m2/repository/techascent/tech.resource/5.04/tech.resource-5.04.jar', '/Users/neil/.m2/repository/com/cognitect/transit-java/0.8.332/transit-java-0.8.332.jar', '/Users/neil/.m2/repository/org/clojure/google-closure-library-third-party/0.0-20201211-3e6c510d/google-closure-library-third-party-0.0-20201211-3e6c510d.jar', '/Users/neil/.m2/repository/org/clojure/core.memoize/1.0.236/core.memoize-1.0.236.jar', '/Users/neil/.m2/repository/org/clojure/tools.analyzer/1.0.0/tools.analyzer-1.0.0.jar', '/Users/neil/.m2/repository/pl/edu/icm/JLargeArrays/1.5/JLargeArrays-1.5.jar', '/Users/neil/.m2/repository/org/roaringbitmap/shims/0.9.0/shims-0.9.0.jar', '/Users/neil/.m2/repository/org/xerial/larray/larray-buffer/0.4.1/larray-buffer-0.4.1.jar', '/Users/neil/.m2/repository/org/clojure/tools.logging/1.1.0/tools.logging-1.1.0.jar', '/Users/neil/.m2/repository/com/fasterxml/jackson/core/jackson-core/2.8.7/jackson-core-2.8.7.jar', '/Users/neil/.m2/repository/commons-codec/commons-codec/1.10/commons-codec-1.10.jar', '/Users/neil/.m2/repository/org/msgpack/msgpack/0.6.12/msgpack-0.6.12.jar', '/Users/neil/.m2/repository/org/clojure/core.cache/1.0.207/core.cache-1.0.207.jar', '/Users/neil/.m2/repository/com/googlecode/json-simple/json-simple/1.1.1/json-simple-1.1.1.jar', '/Users/neil/.m2/repository/org/javassist/javassist/3.18.1-GA/javassist-3.18.1-GA.jar', '/Users/neil/.m2/repository/org/clojure/data.priority-map/1.0.0/data.priority-map-1.0.0.jar']

And I'm getting the following output.

Failed to find class org/cellprofiler/runnablequeue/RunnableQueue

Traceback (most recent call last): File "/Users/neil/Library/Caches/pypoetry/virtualenvs/blisp-97-MdyRi-py3.9/lib/python3.9/site-packages/javabridge/jutil.py", line 327, in start_thread vm.create_mac(args, RQCLS, library_path, libjli_path) File "_javabridge.pyx", line 709, in _javabridge.JB_VM.create_mac RuntimeError: Failed to create Java VM. Return code = -1 Failed to create Java VM

A couple print() statements in javabridge/jutil.py showed me that the start_thread() function is finding library_path and libjli_path, each at:

/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/server/libjvm.dylib /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib

start_thread() is failing at the next line after that, which is vm.create_mac(args, RQCLS, library_path, libjli_path).

Hoping it's something simple I'm doing wrong!

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/LeeKamentsky/python-javabridge/issues/195, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAFBH5LU36FINE5QJ5DSH43UGL5EXANCNFSM5FYO62YQ . Triage notifications on the go with GitHub Mobile for iOS https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675 or Android https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub.

neilyio commented 3 years ago

Hi Lee, thanks for your reply.

I've done some logging inside javabridge/jutil.py inside the start_thread function (defined in start_vm). I placed the following print() calls:

321                library_path = _find_mac_lib("libjvm")
322                libjli_path = _find_mac_lib("libjli")
323                print("library_path =", library_path)
324                print("libjli_path =", libjli_path)

On Eclispe Temurin OpenJDK 8, I get:

library_path = /Library/Java/JavaVirtualMachines/temurin-8.jdk/Contents/Home/jre/lib/server/libjvm.dylib
libjli_path = /Library/Java/JavaVirtualMachines/temurin-8.jdk/Contents/Home/jre/lib/jli/libjli.dylib

On Zulu Azul 11 JDK, I get:

library_path = /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/server/libjvm.dylib
libjli_path = /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib

On Microsoft OpenJDK 17, I get:

library_path = /Library/Java/JavaVirtualMachines/microsoft-17.jdk/Contents/Home/lib/server/libjvm.dylib
libjli_path = /Library/Java/JavaVirtualMachines/microsoft-17.jdk/Contents/Home/lib/libjli.dylib

Same error for all three. I've verified in my file browser that all files exist.

JKelle commented 3 years ago

I'm having the same problem. My computer is an M1 Mac with MacOS Big Sur 11.6. Java-related details of my system:

$ /usr/libexec/java_home -V
Matching Java Virtual Machines (1):
    11.0.13 (arm64) "Azul Systems, Inc." - "Zulu 11.52.13" /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home
/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home

$ echo $JAVA_HOME
/Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home

$ ls -l /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib
-rw-r--r--  1 root  wheel  115584 Oct 16 09:54 /Library/Java/JavaVirtualMachines/zulu-11.jdk/Contents/Home/lib/jli/libjli.dylib

When I run this code:

import javabridge
import bioformats
javabridge.start_vm(class_path=bioformats.JARS, run_headless=True)

I get this error:

Failed to open libjli.dylib.

Traceback (most recent call last):
  File "/Users/jkelle/opt/miniconda3/envs/worker/lib/python3.8/site-packages/javabridge/jutil.py", line 282, in start_thread
    vm.create_mac(args, RQCLS, library_path, libjli_path)
  File "_javabridge.pyx", line 709, in _javabridge.JB_VM.create_mac
RuntimeError: Failed to create Java VM. Return code = -1
Failed to create Java VM
Traceback (most recent call last):
  File "temp.py", line 4, in <module>
    javabridge.start_vm(class_path=bioformats.JARS, run_headless=True)
  File "/Users/jkelle/opt/miniconda3/envs/worker/lib/python3.8/site-packages/javabridge/jutil.py", line 319, in start_vm
    raise RuntimeError("Failed to start Java VM")
RuntimeError: Failed to start Java VM

I don't know why it failed to open libjli.dylib.

JKelle commented 3 years ago

I was using a conda environment, and I installed javabridge and bioformats into that conda environment via pip. I was able to resolve this issue by installing openjdk via conda (conda install openjdk) and unsetting my JAVA_HOME env variable.

neilyio commented 3 years ago

@JKelle That's really helpful, thanks!

Could you provide some details about your new conda-based installation? Do you know where libjli.dylib and libjvm.dylib are located? Do you know if your conda environment is setting JAVA_HOME to a different value?

JKelle commented 3 years ago

@neilyio When I run os.getenv("JAVA_HOME") in my conda environment, I get '/Users/jkelle/opt/miniconda3/envs/worker' ('worker' is the name of my conda environment).

When I run the code in _find_mac_lib, javabridge finds libjli.dylib at /Users/jkelle/opt/miniconda3/envs/worker/lib/jli/libjli.dylib

neilyio commented 3 years ago

Thanks for checking on that. I wonder what it is about conda's OpenJDK that makes it suddenly work.

LeeKamentsky commented 3 years ago

Thanks for looking into this guys.