search

imagej / pyimagej

Use ImageJ from Python
https://pyimagej.readthedocs.io/
Other
474 stars 83 forks source link

Indefinite hanging when trying to intialize pyimagej with `ij = imagej.init()` #214

Open metersk opened 2 years ago

metersk commented 2 years ago

I'm having some issues with initialization in a Docker container. ij = imagej.init() will hang indefinitely. I have not been successful yet in getting it to initialize. Here are the debug statements that I see

 => [8/8] RUN python /opt/fiji/init_fiji.py                                                         906.0s
 => => # Returning expanded coordinate io.scif:scifio-labeling:0.3.1.
 => => # Returning expanded coordinate net.imglib2:imglib2-imglyb:1.0.1.
 => => # First time start-up may be slow. Downloaded dependencies will be cached for shorter start-up time
 => => # s in subsequent executions.
 => => # Executing: ('/opt/conda/bin/mvn', '-B', '-f', '/root/.jgo/io.scif/scifio-labeling/0.3.1/eb3228841
 => => # e43c0aa0cffa924ea6aa524384a3685113d54510c0ed985c20d2ab0/pom.xml', 'dependency:resolve')

Here is my Dockerfile:

FROM --platform=linux/amd64 continuumio/miniconda3:4.11.0

ENV CONDA_DIR=/opt/conda
ENV JAVA_HOME=/opt/conda/jre

RUN conda install mamba -n base -c conda-forge

RUN mamba install openjdk=8 pyimagej unzip \
    -c conda-forge -y &&\
    mamba clean --all -f -y

RUN mamba create -n pyimagej -c conda-forge pyimagej openjdk=8 unzip

WORKDIR /opt/fiji

RUN wget -q https://downloads.imagej.net/fiji/latest/fiji-linux64.zip \
    && unzip fiji-linux64.zip \
    && rm fiji-linux64.zip

ENV PATH $PATH:/opt/fiji/Fiji.app

COPY init_fiji.py .
RUN python /opt/fiji/init_fiji.py

And here is the init_fiji.py script:

import imagej
import imagej.doctor

imagej.doctor.debug_to_stderr()
ij = imagej.init("/opt/fiji/Fiji.app")
print(f"ImageJ version: {ij.getVersion()}")

If I swap ij = imagej.init("/opt/fiji/Fiji.app") for ij = imagej.init(), these are the debug statemetns I get:

 => [8/8] RUN python /opt/fiji/init_fiji.py                                   69.7s
 => => # Returning expanded coordinate net.imglib2:imglib2-imglyb:1.0.1.
 => => # First time start-up may be slow. Downloaded dependencies will be cached fo
 => => # r shorter start-up times in subsequent executions.
 => => # Executing: ('/opt/conda/bin/mvn', '-B', '-f', '/root/.jgo/net.imagej/image
 => => # j/RELEASE/bf9da2ebe6c78d8bd105a97ce32adcfe853ce5e97dd621b9e35936d1c6abf90c
 => => # /pom.xml', 'dependency:resolve')
metersk commented 2 years ago

Okay, the plot thickens...

With this Dockerfile I get the following error:

FROM --platform=linux/amd64 continuumio/miniconda3:4.11.0

ENV CONDA_DIR=/opt/conda
ENV JAVA_HOME=/opt/conda/jre

RUN conda install mamba -n base -c conda-forge

RUN mamba install openjdk pyimagej unzip \
    -c conda-forge -y &&\
    mamba clean --all -f -y

RUN mamba create -n pyimagej -c conda-forge pyimagej openjdk unzip

WORKDIR /opt/fiji

RUN wget -q https://downloads.imagej.net/fiji/latest/fiji-linux64.zip \
    && unzip fiji-linux64.zip \
    && rm fiji-linux64.zip

ENV PATH $PATH:/opt/fiji/Fiji.app

COPY test.py .

RUN python -c 'import imagej; import imagej.doctor; imagej.doctor.debug_to_stderr(); ij = imagej.init(); print(f"ImageJ version: {ij.getVersion()}")'

Error:

❯ docker build -t rando .
[+] Building 3.1s (12/12) FINISHED
 => [internal] load build definition from Dockerfile                           0.0s
 => => transferring dockerfile: 807B                                           0.0s
 => [internal] load .dockerignore                                              0.0s
 => => transferring context: 2B                                                0.0s
 => [internal] load metadata for docker.io/continuumio/miniconda3:4.11.0       0.2s
 => [1/8] FROM docker.io/continuumio/miniconda3:4.11.0@sha256:24103733efebe6d  0.0s
 => [internal] load build context                                              0.0s
 => => transferring context: 29B                                               0.0s
 => CACHED [2/8] RUN conda install mamba -n base -c conda-forge                0.0s
 => CACHED [3/8] RUN mamba install openjdk pyimagej unzip     -c conda-forge   0.0s
 => CACHED [4/8] RUN mamba create -n pyimagej -c conda-forge pyimagej openjdk  0.0s
 => CACHED [5/8] WORKDIR /opt/fiji                                             0.0s
 => CACHED [6/8] RUN wget -q https://downloads.imagej.net/fiji/latest/fiji-li  0.0s
 => CACHED [7/8] COPY test.py .                                                0.0s
 => ERROR [8/8] RUN python -c 'import imagej; import imagej.doctor; imagej.do  2.8s
------
 > [8/8] RUN python -c 'import imagej; import imagej.doctor; imagej.doctor.debug_to_stderr(); ij = imagej.init(); print(f"ImageJ version: {ij.getVersion()}")':
#12 2.559 Traceback (most recent call last):
#12 2.559   File "<string>", line 1, in <module>
#12 2.560   File "/opt/conda/lib/python3.9/site-packages/imagej/__init__.py", line 1496, in init
#12 2.563     success = _create_jvm(ij_dir_or_version_or_endpoint, mode, add_legacy)
#12 2.563   File "/opt/conda/lib/python3.9/site-packages/imagej/__init__.py", line 1603, in _create_jvm
#12 2.565     if hasattr(sj, "jvm_version") and sj.jvm_version()[0] >= 9:
#12 2.565   File "/opt/conda/lib/python3.9/site-packages/scyjava/__init__.py", line 55, in jvm_version
#12 2.565     default_jvm_path = jpype.getDefaultJVMPath()
#12 2.565   File "/opt/conda/lib/python3.9/site-packages/jpype/_jvmfinder.py", line 74, in getDefaultJVMPath
#12 2.565     return finder.get_jvm_path()
#12 2.565   File "/opt/conda/lib/python3.9/site-packages/jpype/_jvmfinder.py", line 212, in get_jvm_path
#12 2.566     raise JVMNotFoundException("No JVM shared library file ({0}) "
#12 2.566 jpype._jvmfinder.JVMNotFoundException: No JVM shared library file (libjvm.so) found. Try setting up the JAVA_HOME environment variable properly.
------
executor failed running [/bin/sh -c python -c 'import imagej; import imagej.doctor; imagej.doctor.debug_to_stderr(); ij = imagej.init(); print(f"ImageJ version: {ij.getVersion()}")']: exit code: 1

BUT if I remove the last line from the Dockerfile RUN python -c 'import imagej..... and build and then enter the container with docker run -it rando /bin/bash I can run this from the command line:

python -c 'import imagej; import imagej.doctor; imagej.doctor.debug_to_stderr(); ij = imagej.init(); print(f"ImageJ version: {ij.getVersion()}")'

The first time i run this from a freshly build image it will usually hang on something, so I exit with ctrl+c and rerun and then it usually initializes successfully.

Would love to figure out how to do this all in the image so that I don't need to enter the container and execute commands manually.

ctrueden commented 1 year ago

Sorry @metersk for not replying till now.

Any imagej.init call using Maven coordinates requires Internet access to download the JAR files from the remote repository. Is your Docker container configured to allow such network access? My understanding of Docker is that those types of actions are often locked down, unless you specifically allow it. Right? If your container is not allowed network access, this could explain the issue.

So, the idea here is that you do the initial imagej.init() in the Dockerfile while building the container, so that all the remote JARs are already downloaded, maybe? And then imagej.init() should not need the network to initialize while the container is running? I think this idea is sound, but in practice, the imagej.init() call, which invokes mvn, may attempt to do some networking things anyway by default. You could try adding -o to the mvn command, which tells Maven to run in offline mode, but I think you might have to hack jgo right now to add -o to the list of Maven arguments. Better would be a PR to jgo exposing a configuration option for offline mode, and/or additional Maven options more generally...

Might be simpler to just open your Docker container up to network access, if you can tolerate that security-wise.