Failed to run test installation with openjdk8 on Mac M1

[clementhelsens]

Hello! I just got a fresh Mambaforge for my macbook (M1 chip) following the instructions here:

created and activated the environment

mamba create -n pyimagej pyimagej openjdk=8
mamba activate pyimagej

but running the suggested test python -c 'import imagej; ij = imagej.init("2.5.0"); print(ij.getVersion())'

fails with this error

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/clementhelsens/mambaforge/envs/pyimagej/lib/python3.10/site-packages/imagej/__init__.py", line 1191, in init
    success = _create_jvm(ij_dir_or_version_or_endpoint, mode, add_legacy)
  File "/Users/clementhelsens/mambaforge/envs/pyimagej/lib/python3.10/site-packages/imagej/__init__.py", line 1304, in _create_jvm
    if hasattr(sj, "jvm_version") and sj.jvm_version()[0] >= 9:
  File "/Users/clementhelsens/mambaforge/envs/pyimagej/lib/python3.10/site-packages/scyjava/_java.py", line 140, in jvm_version
    version = subprocess.check_output(
  File "/Users/clementhelsens/mambaforge/envs/pyimagej/lib/python3.10/subprocess.py", line 420, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/Users/clementhelsens/mambaforge/envs/pyimagej/lib/python3.10/subprocess.py", line 524, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/clementhelsens/mambaforge/envs/pyimagej/jre/bin/java', '-version']' died with <Signals.SIGKILL: 9>.

The doc says it should work fine for python3.10 except for windows, but is this also expected for Mac? Thanks!

[NicoKiaru]

Not sure it will solve anything, but can you retry with ij = imagej.init("2.9.0");, because that's the current version

[clementhelsens]

indeed, exact same error with current version. will give it a try tomorrow with older python versions

[ctrueden]

@clementhelsens Thanks for the report, and sorry for the trouble. It works on my Intel macOS system with Python 3.10.6 and openjdk 1.8.0_332 from conda-forge, so I'm guessing this is an issue with the M1 version of openjdk on conda. You could try installing another OpenJDK and setting JAVA_HOME to point to it before running the the test?

[clementhelsens]

hello @ctrueden, with openjdk free from version, it install 17.0.3 and breaks with this error:

[ERROR] Cannot create plugin: org.scijava.plugins.scripting.javascript.JavaScriptScriptLanguage
2.5.0/1.53r

with openjdk=11 it seems I can get the expected printout

2.5.0/1.53r

Looks like you are right, there is an issue with openjdk=8 from conda-forge, from mamba create:

+ openjdk               8.0.332  he4db4b2_0            conda-forge/osx-arm64       45MB`

I am completely new to image processing, but will get significantly involved from now on, thus apologise for the naive questions, but is there a reason to stick to such old openjdk? It seems from here that version 8 is ~10 years old.

[NicoKiaru]

I am completely new to image processing, but will get significantly involved from now on, thus apologise for the naive questions, but is there a reason to stick to such old openjdk? It seems from here that version 8 is ~10 years old.

Here's what I understood:

First of all, Java 8 is an LTS version supported until 2030 (https://www.oracle.com/java/technologies/java-se-support-roadmap.html). It's still heavily used. There are regular updates, bug fixes and even backports sometimes.

8 looks like a small number compared to 17, but that's because the release frequency of Java increased a lot in recent years. Their went from 3 year between releases to one release every 6 monthes (visible on the roadmap). Java available LTS versions are 8, 11 and 17.

Java 11 support is planned until 2026. As far as I understand Fiji can already work with it, but there's some work that needs to be done to integrate it smoothly in the different OS launcher (needed for when you download Fiji from https://fiji.sc/). When using Fiji from pyimagej, this is not an issue apparently.

Just to confirm: your install is working now with Java 11, right ?

For Java 11 and above, some restrictions were introduced on how the different java packages can interact with one another. This creates issues with how the scijava framework is currently built. Scijava is the framework, on top of which some parts of Fiji is built. That's why some work is currently undertaken to revisit and make a new framework that'll be compatible with Java 11+ (https://github.com/scijava/incubator).

Some extra information is available in https://imagej.net/learn/faq, but I think this page needs a refresh (it dates back from the time Java 6 was still supported). @ctrueden will probably want to weigh in as well.

[ctrueden]

@NicoKiaru is :100:% right with everything he said. PyImageJ works well with either OpenJDK 8 or 11. So since OpenJDK 11 is working for you @clementhelsens, I advise you stick with that. There are currently problems with OpenJDK 17 as you noted; although the JavaScript error is non-fatal, it just means that JavaScript scripting doesn't work with OpenJDK 17 because OpenJDK 17 no longer ships a JavaScript script engine (Nashorn was removed in OpenJDK 14). So depending on what you are using PyImageJ for, OpenJDK 17 might also work OK for you.

As for OpenJDK 11 versus OpenJDK 8, the main downside is that the JavaFX library underwent breaking changes from 8 to 11. So if you are using any ImageJ plugins that depend on JavaFX, they won't work with OpenJDK 11 because they are all currently written against JavaFX 8. One must use JavaFX 11 with OpenJDK 11, or JavaFX 8 with OpenJDK 8 (for the latter I recommend the Zulu JDK+FX 8 flavor). Whereas with OpenJDK 11 + JavaFX 11, JavaFX is now a separate set of libraries that can be used as normal dependencies. But because the API underwent breaking changes, we need to update all plugins together as we transition ImageJ2 and Fiji from Java 8 to Java 11+. It was work my team had planned to do this year, but unfortunately we have too many other urgent priorities right now to tackle it. So in the meantime: OpenJDK 8 is the most recommended version.

[clementhelsens]

Thanks @ctrueden @NicoKiaru for the detailed answers! Need to get more familiar with the Java ecosystem now :) Think the issue could be closed now, I renamed it to reflect what the issue was, but please feel free to change!

[ctrueden]

A couple of quick ideas for making this better for Mac M1 users:

1. Improve the imagej.doctor to detect and/or test for unsupported OS/arch/JDK combinations.
2. Catch the CalledProcessError in scyjava and report that the OpenJDK installation is broken.

[ctrueden]

(1) I checked, and the PyImageJ doctor already runs java -version; I'm curious what running the checkup() command would say on an M1 Mac with the default openjdk 8 from conda-forge.

(2) With scijava/scyjava@4409ef34336e95e6db8574cf0100b79f7352f13d, scyjava now catches CalledProcessError and raises RuntimeError instead, so that PyImageJ can recover from the failed Java version check, which might let conda-forge's openjdk v8 work on Mac M1 (testing needed).

[clementhelsens]

Hi @ctrueden,

(1)

Checking Python:
--> Python executable = /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/bin/python

Checking environment:
--> CONDA_PREFIX = /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8
--> Python executable matches Conda environment.

Checking Python dependencies:
--> jgo: /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/lib/python3.10/site-packages/jgo/__init__.py
--> scyjava: /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/lib/python3.10/site-packages/scyjava/__init__.py
--> imglyb: /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/lib/python3.10/site-packages/imglyb/__init__.py
--> pyimagej: /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/lib/python3.10/site-packages/imagej/__init__.py

Checking Maven:
--> Maven executable = /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/bin/mvn
$ mvn -v
Command '['/Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/bin/mvn', '-v']' died with <Signals.SIGKILL: 9>.

Checking Java:
--> JAVA_HOME = /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8
--> Java executable = /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/bin/java
$ java -version
Command '['java', '-version']' died with <Signals.SIGKILL: 9>.

Great job! All looks good.

if I try to run java from the shell it dies zsh: killed java

which java on my mamba environment with openjdk8 gives: /Users/clementhelsens/mambaforge/envs/pyimagej_openjdk8/bin/java

while with openjdk11 points to /usr/bin/java not sure why it does not point to my virtual env...

I should also point out that I am new to the Java ecosystem :)

[ctrueden]

Thanks @clementhelsens, that confirms that something is weird with the installed openjdk from conda-forge on that system. I am very amused by:

$ java -version
Command '['java', '-version']' died with <Signals.SIGKILL: 9>.

Great job! All looks good.

Computers are so dumb. :grin:

[ctrueden]

See also #267, #268