How to safely use pyimagej and napari-imagej within a Python script

[kephale]

As promised in https://github.com/imagej/napari-imagej/issues/241

It looks like there is a threading/non-blocking situation that arises when using pyimagej and napari-imagej within the same script. It seems like napari-imagej is spinning up the JVM in a separate thread[?] and not blocking when the JVM is being launched, as a result the Python evaluation continues and subsequent jimport calls cannot work yet.

This script does not work:

import napari
from napari_imagej.widgets.napari_imagej import NapariImageJWidget

from napari_imagej import settings

endpoint = "sc.fiji:fiji:2.13.0+org.morphonets:SNT:MANAGED"

settings.imagej_directory_or_endpoint = endpoint

if __name__ == "__main__":

    viewer = napari.Viewer()

    widget = NapariImageJWidget(viewer)

    viewer.window.add_dock_widget(widget)

    from scyjava import jimport
    MouseLightLoader = jimport('tracing.io.MouseLightLoader')

and gives the following error:

Traceback (most recent call last):
  File "<string>", line 17, in __PYTHON_EL_eval
  File "/Users/kharrington/nesoi/examples/napari_imagej_snt.py", line 19, in <module>
    MouseLightLoader = jimport('tracing.io.MouseLightLoader')
  File "/Users/kharrington/mambaforge/envs/nesoi/lib/python3.10/site-packages/scyjava/_java.py", line 405, in jimport
    return jpype.JClass(class_name)
  File "/Users/kharrington/mambaforge/envs/nesoi/lib/python3.10/site-packages/jpype/_jclass.py", line 99, in __new__
    return _jpype._getClass(jc)
TypeError: Class tracing.io.MouseLightLoader is not found

and displays Error: Invalid ImageJ2 in the search bar. which seems to have been triggered by the jpype error above even though it is not exactly an invalid ImageJ2 situation.

[gselzer]

Yeah, I'd be happy to get to a better solution here @kephale.

The napari-imagej widget spins up the JVM on a separate QThread to avoid bricking the napari application for seconds to minutes (it's an especially long time if jgo has to download ImageJ). But of course then we have the downside that you are seeing now.

To make something like this work, we have a couple of options:

1. Add a synchronous call to napari_imagej.java.init_ij() before the line widget = NapariImageJWidget(viewer). I tried this in your script, and it fails with the same message you saw in #211. @ctrueden is this a jgo bug:

   
   Traceback (most recent call last):
   File "/home/g/code/imagej/napari-imagej/src/napari_imagej/widgets/napari_imagej.py", line 230, in run
   init_ij()
   File "/home/g/code/imagej/napari-imagej/src/napari_imagej/java.py", line 77, in init_ij
   _ij = imagej.init(**_configure_imagej())
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/imagej/__init__.py", line 1200, in init
   success = _create_jvm(ij_dir_or_version_or_endpoint, mode, add_legacy)
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/imagej/__init__.py", line 1424, in _create_jvm
   sj.start_jvm()
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/scyjava/_java.py", line 190, in start_jvm
   _, workspace = jgo.resolve_dependencies(
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/jgo/jgo.py", line 701, in resolve_dependencies
   link(
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/jgo/jgo.py", line 215, in link
   raise e
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/jgo/jgo.py", line 212, in link
   return link(source=source, link_name=link_name, link_type="hard")
   File "/media/d/mambaforge/envs/napari-imagej-dev/lib/python3.10/site-packages/jgo/jgo.py", line 207, in link
   return os.link(source, link_name)
   FileNotFoundError: [Errno 2] No such file or directory: '/home/g/.m2/repository/org/jzy3d/jzy3d-native-jogl-core/2.2.1/jzy3d-native-jogl-core-2.2.1-tests.test-jar' -> '/home/g/.jgo/sc.fiji/fiji/2.13.0/6645db0bfd2a4bea3de6900b5d8bc810a00dba1bf1af757a848f3d136ce3a862/jzy3d-native-jogl-core-2.2.1-tests.test-jar'

2. We could provide some fully synchronous way of initializing `NapariImageJWidget` - two different pathways is suboptimal, but at least we wouldn't brick napari in the general case. Do you know if the napari team has talked about asynchronous instantiation of napari widgets, @kephale? Because if we did that then we may not need this additional `QThread` in napari-imagej.

3. Something else? @ctrueden do you have opinions?

[ctrueden]

I think the solution is (1). Scripts should demand synchronous initialization of napari-imagej before doing anything else. The jgo bug with test-jar packaging was supposed to have been fixed in commit scijava/jgo@49aa492aa4718608d59f0cbc0d3d7810f6398a5b, included in jgo 1.0.5. Is it possible you are running an older version of jgo?

Edit: It looks like jgo 1.0.5 never made it to conda-forge, because I forgot to merge the PR until now. For the moment, you can work around using pip install jgo==1.0.5 to upgrade. But hopefully conda-forge will update shortly.

[gselzer]

Ah, okay, thanks for merging that PR @ctrueden. I don't get the jgo bug anymore.

@kephale by adding in the init_ij call, and by fixing the MouseLightLoader import, the script runs without error for me. What do you think of this? Can you think of a better way to synchronously initialize ImageJ without bricking the napari viewer?

[kephale]

2. Do you know if the napari team has talked about asynchronous instantiation of napari widgets, @kephale?

There is discussion about supporting custom initialization code for plugins. The old plugin framework supported it, but it isn't present in npe2 at the moment.

[gselzer]

@kephale is there anything else that you'd like done here?

[kephale]

Nope, I'm happy, thank you!