PyImageJ in JupyterLab_Installation_and_Image_dims_Tech_details

[xp1632]

Installation of PyImageJ by mamba

https://py.imagej.net/en/latest/Install.html

1. Install mamba in conda by: conda install mamba

2. Install Miniforge3: https://github.com/conda-forge/miniforge#miniforge3

3. Install PyImageJ via: mamba create -n pyimagej -c conda-forge pyimagej openjdk=11

4. activate PyImageJ environment in mamba activate pyimagej

5. To test if pyimagej is installed, we run python -c "import imagej; ij = imagej.init('2.14.0'); print(ij.getVersion())" (in original guide it says : python -c 'import imagej; ij = imagej.init("2.14.0"); print(ij.getVersion())', however the double quote and single quote here would lead to syntax error)

6. in pyimageJ environment, we jupyter lab

[xp1632]

Trouble shooting of cannot find imageJ module in jupyter

• After running jupyter lab in the pyimageJ environment, the imageJ module is not found
• Reasons could be:

The error message suggests that the Jupyter Lab is not able to find the imagej module. This could be because Jupyter Lab is not using the correct Python interpreter from the pyimagej environment.

Here are a few steps to troubleshoot this issue:

1. Check the Python interpreter in Jupyter Lab: In a new Jupyter Lab cell, run the following command:

   import sys
   print(sys.executable)

   This command prints the path to the Python interpreter that Jupyter Lab is using. Make sure that this path points to the Python interpreter in the pyimagej environment.

2. Install the ipykernel package in the pyimagej environment: The ipykernel package allows Jupyter to create Python kernels for different environments. You can install it using the following command:

   mamba install -n pyimagej ipykernel

3. Create a new Jupyter kernel for the pyimagej environment: After installing ipykernel, you can create a new Jupyter kernel for the pyimagej environment using the following command:

   python -m ipykernel install --user --name pyimagej --display-name "Python (pyimagej)"

   This command creates a new Jupyter kernel named "Python (pyimagej)".

4. Select the new Jupyter kernel in Jupyter Lab: After creating the new Jupyter kernel, you need to select it in Jupyter Lab. You can do this by clicking on the kernel name in the top right corner of Jupyter Lab and selecting "Python (pyimagej)" from the dropdown menu.

After following these steps, try running your code again in Jupyter Lab. It should now be able to find the imagej module.

[xp1632]

Caused by different python interpreters:

The Python interpreters are different when we run commands in cmd pyimagej env and the jupyter lab we run in it

---

• For different python environments, we need to install ipykernel and specify a new jupyter kernel for it.

1. mamba activate pyimagej
2. mamba install ipykernel
3. python -m ipykernel install --user --name pyimagej --display-name "Python (pyimagej)"
4. Select the new Jupyter kernel in Jupyter Lab

---

Install Pyimagej in base env?

• We could also ,not creat a new env for pyimagej ,and just install it into our env by mamba install -c conda-forge pyimagej openjdk=11

• But this is not recommended, it's generally recommended to create a new environment for each project to avoid conflicts between package versions.

[xp1632]

Installation of PyimageJ done

[xp1632]

PyimageJ mechanism:

PyImageJ is built on the scyjava library, which is built on JPype and jgo. These libraries enable us to import any available Java class into our Python program, and use its full API, just as we could from within a Java program.

---

• pyimagej uses jgo internally to call up ImageJ
• jgo can launch java code from commandline
• JPype is a python module to provide full access to Java from within Python
• Scyjava is built on jgo + Jpype:
• it provides supercharged Java access from Python.
• we can use jimport to import java library in Python

[xp1632]

import imagej java classes in python:

• As we stated before, we can use scyjava.jimport to import java classes in python

- Here's an example of

• load a local image to imagej
• show it with ij.py.show()
• importing HyperSphereShape java class to python
• and run a mean op filter on the image:

import imagej

# initialize ImageJ
ij = imagej.init('sc.fiji:fiji:2.14.0')
print(f"ImageJ version: {ij.getVersion()}")

# load test image
dataset = ij.io().open('test_image.tif')

# display test image (see the Working with Images for more info)
ij.py.show(dataset)

# import HyperSphereShape and create radius of 5
HyperSphereShape = jimport('net.imglib2.algorithm.neighborhood.HyperSphereShape')
radius = HyperSphereShape(5)

# apply filter
result = ij.dataset().create(dataset)
ij.op().filter().mean(result, dataset, radius)

# show image
ij.py.show(result)

[xp1632]

Convenient methods of PyImageJ

Ref: Doc5	ij.py.	function	more information
show	Show an image	06-Working-with-Images
to_java	Convert data from Python to Java	03-Sending-Data-to-Java
from_java	Convert data from Java to Python	04-Retrieving-Data-from-Java
run_macro	Run an original ImageJ macro	07-Running-Macros-Scripts-and-Plugins
run_script	Run an ImageJ script (supported languages)	07-Running-Macros-Scripts-and-Plugins
run_plugin	Run a plugin	07-Running-Macros-Scripts-and-Plugins
initialize_numpy_image	Create a new numpy image in the same shape as input image	06-Working-with-Images
sync_image	Synchronize data between ImageJ and ImageJ2 data structures	--
active_dataset	Get the active image as a Dataset	--
active_xarray	Get a copy of the active image as an xarray.DataArray	--
active_imageplus	Get the ImagePlus from the current window	--

[xp1632]

Warning messages for ImageJ1's RoiManager/WindowsManager

Ref: Doc5

• PyImageJ offers IJ1 legacy check by print(f"Legacy layer active: {ij.legacy.isActive()}")
• And the classes that have limited functions in headless mode will also print a warning:

[xp1632]

Rest plan of notebook topics:

• We quickly scanned the 12 notebooks and summarize the things that are useful for us in topics

• [x] 2.0 ! important Image data types in PyImageJ in Doc 6 https://github.com/imagej/pyimagej/blob/main/doc/06-Working-with-Images.ipynb

• This notebooks tell details of image channels

• 6.1, 6.2, 6.6 are important

• [ ] 2.1 - ImageJ1 plugin Doc11: https://github.com/imagej/pyimagej/blob/main/doc/11-Working-with-the-original-ImageJ.ipynb

  Doc8: https://github.com/imagej/pyimagej/blob/main/doc/08-Discover-and-run-ImageJ-commands.ipynb

• [ ] 2.2 - ImageJ2 ops

  • basic ops are in Doc 10: https://github.com/imagej/pyimagej/blob/main/doc/10-Using-ImageJ-Ops.ipynb
  • more complex op use cases are in : https://py.imagej.net/en/latest/imagej-ops-use-cases.html

• [x] 2.3 - customized plugins

  • could only be defined in jar and add to classpath before imagej.init()
  • reference: https://github.com/imagej/pyimagej/blob/main/doc/Initialization.md#plugins-without-maven-endpoints
  • more specifically, we need to check the _set_ij_env function in PyimageJ/src/imagej/__init__.py

  def _set_ij_env(ij_dir):
  """
  Create a list of required jars and add to the java classpath.
  
  :param ij_dir: System path for Fiji.app.
  :return: num_jar(int): Number of jars added.
  """
  jars = []
  # search jars directory
  jars.extend(find_jars(ij_dir + "/jars"))
  # search plugins directory
  jars.extend(find_jars(ij_dir + "/plugins"))
  # add to classpath
  sj.config.add_classpath(os.pathsep.join(jars))
  return len(jars)

  • The add_classpath function is where we add jars to current classpath

• [x] 2.4 - marcos and scripts

  • Doc 7: https://github.com/imagej/pyimagej/blob/main/doc/07-Running-Macros-Scripts-and-Plugins.ipynb

• [x] 2.5 - Conversion Python -> Java

• document 3: https://github.com/imagej/pyimagej/blob/main/doc/03-Sending-Data-to-Java.ipynb

• And doc6: C6.7 : https://github.com/imagej/pyimagej/blob/main/doc/06-Working-with-Images.ipynb

• [x] 2.6 - Conversion Java ->Python

• document 4: https://github.com/imagej/pyimagej/blob/main/doc/04-Retrieving-Data-from-Java.ipynb

• [ ] 2.7 blob example and segmentation usecase: https://py.imagej.net/en/latest/segmentation-use-cases.html

[xp1632]

Images data types in PyImageJ:

---

Two ways of Open Images:

• Images we get from ij.io().open() returns ImageJ2 Dataset java object.

• Image we get from ij.IJ.openImage() legacy opener returns ImageJ1 ImagePlus java object

• Unlike the ImageJ we used before that shows imageJ in the next cell immediately

• PyImageJ's open only read Image, we need another function to show it because now we are in Python environment

---

Or we could read image by python package skimage:

# Import an image with scikit-image.
# NB: 4D (X, Y, Channel, Z) confocal image of a HeLa cell in metaphase.
import skimage
url = 'https://media.imagej.net/pyimagej/4d/metaphase_hela_cell.tif'
img = skimage.io.imread(url)

# get microtubule slice
# NB: The dimension shape of `img` is (pln, row, col, ch) or (Z, Y, X, Channel).
mt = img[0, :, :, 2]

# show image
ij.py.show(mt)

---

Two ways of Display images:

• ij.ui().show() in PyImageJ's interactive or gui mode which calls ImageJ's image viewer

  • can display Java object, n-dimensional image adta

• ij.py.show() which calls matplotlib's pyplot

  • can display Python object, 2D image data

• in source code __init.py__/show(), we can see

  • if it's a java object, we first convert it into a Python numpy array via from_java
  • Then show it with matplotlib's pyplot

• pyplot.show

• pyplot.imshow

[xp1632]

images can also be displayed with ipywidgets, napari, itkwidgets

• But it's not compatible with VP project, so we don't talk in details of 6.3-6.5

[xp1632]

Details about image data dims:

Ref: Doc6.6

---

• Take away for dims:

• ImageJ2 Dataset and xArray has different orders of dimensions

• ImageJ images has dims as [X,Y,Channel,(Z),Time]

• xArray images has dims as [t,(pln),row,col,ch], where row and col correspond to Y and X, pln <->Z

• the dtype for both images are UnsignedShortType which is unint16 with integer values in [0,65535]

---

• The sample test_timeseries.tif is a 4D(X,Y,Channel,Time) dataset.
• We can not display it by ij.py.show() because pyplot only support 2d image here

---

• we write a function to print details of an image object:

def dump_info(image):
    """A handy function to print details of an image object."""
    name = image.name if hasattr(image, 'name') else None # xarray
    if name is None and hasattr(image, 'getName'): name = image.getName() # Dataset
    if name is None and hasattr(image, 'getTitle'): name = image.getTitle() # ImagePlus
    print(f" name: {name or 'N/A'}")
    print(f" type: {type(image)}")
    print(f"dtype: {image.dtype if hasattr(image, 'dtype') else 'N/A'}")
    print(f"shape: {image.shape}")
    print(f" dims: {image.dims if hasattr(image, 'dims') else 'N/A'}")

dump_info(dataset2)

# convert the ImageJ image (Java) to an xarray.DataArray (Python)
xarr = ij.py.from_java(dataset2)

# dump info on xarray.DataArray result from ij.py.from_java()
dump_info(xarr)

---

• We can see the orders of dimensions are different in xArray datatype and imageJ2 dataset datatype

[xp1632]

Conversion: Python <----> Java:

• Here's a conversion type list between Python and Java:

Python type	Java type	Notes
numpy.ndarray	net.imglib2.img.Img	An Img is both a RandomAccessibleInterval and an IterableInterval.
xarray.DataArray	net.imagej.Dataset	A Dataset is an Img with additional metadata including dimensional axis labels.
int	Integer, Long, or BigInteger	Destination Java type depends on magnitude of the integer.
float	Float, Double, or BigDecimal	Destination Java type depends on magnitude and precision of the value.
str	String
bool	boolean
dict	LinkedHashMap	Converted recursively (i.e., elements of the dict are converted too).
set	LinkedHashSet	Converted recursively (i.e., elements of the set are converted too).
list	ArrayList	Converted recursively (i.e., elements of the list are converted too).

---

Passing data from Python to Java:

• ij.py.to_java() is capable of converting common Python and numpy data types -----> their Java/ImageJ/ImageJ2 equivalent

• converting numpy array or xarray to JAVA creates a java object and this java object points to the numpy array. '

• This means changing the Java object also changes the numpy array

• In short, change java object also changes linked numpy array or xarray

---

Python List --> Java List:

Ref:Doc3

# create lists
python_list = [1, 2, 3, 4]
java_list = ij.py.to_java(python_list)

# modify one list
python_list[0] = 4

# check list contents
print(f"python_list: {python_list}\njava_list: {java_list}")

• we can see the lists are independent from each other

---

[xp1632]

5.2 Numpy array --> Java RandomAccessibleInterval

import numpy as np

# get numpy array and list
test_arr = np.array([[5, 12], [21, 32]])
test_list = [1, 2, 4, 8, 16, 32, 64]

# convert array and list to Java
jarr = ij.py.to_java(test_arr)
jlist = ij.py.to_java(python_list)

print(type(jarr))
print(type(jlist))

• Numpy array is converted to RandomAccesibleInterval which is wrapped as Java DefaultDataset
• 

[xp1632]

5.3 Passing Image from Python to Java

---

Take away:

- We need to pay attention that:

1. Modifying the java object we get from to_java() method also modifies its linked python object
2. Image Data dimensions has different orders: ImaeJ2(XYCZ), scikit(ZCYX)

---

# Import an image with scikit-image.
# NB: 4D (X, Y, Channel, Z) confocal image of a HeLa cell in metaphase.
import skimage

url = 'https://media.imagej.net/pyimagej/4d/metaphase_hela_cell.tif'
img = skimage.io.imread(url)
# get microtubule slice
# NB: The dimension shape of `img` is (pln, row, col, ch) or (Z, Y, X, Channel).
mt = img[0, :, :, 2]

# show image
ij.py.show(mt)

• Op that requires a RandomAccessibleInterval can run on a NumPy array that has been passed to to_java().
• This method creates a view and modifies the underlying Python object:

result = np.zeros(mt.shape)
# these sigmas will be nice for the larger sections
sigma1 = 2.5
sigma2 = 0.5
# note the use of to_java on img and result to turn the numpy images into RAIs
ij.op().filter().dog(ij.py.to_java(result), ij.py.to_java(mt), sigma1, sigma2)
# purple highlights the edges of the vessels, green highlights the centers
ij.py.show(result, cmap = 'viridis')

[xp1632]

ndarray and xarray:

• As we stated in #63 :

• ndarray doesn't have labeled dims as xarray does

• 

• we can construct a xarray based on ndarray by including correct dimensional axis labels:

• The convention of dims order in scikit image is

• 

• which is equilvalant to (T,Z,Y,X,C)

[xp1632]

5.4 Conversion Java -> Python

- To translate a Java image into Python, use:

• python_image = ij.py.from_java(java_image)

• ij2 Dataset that has metadata --> xarrays

• RAI or ImagePlus that are metadata-free ---> numpy array

• We can see the type of java object before and after the conversion:

  
  colony_img= ij.io().open('https://wsr.imagej.net/images/Cell_Colony.jpg')

p_colony_img = ij.py.from_java(colony_img) ij.py.show(p_colony_img,cmap='gray') print(f"cell_colony type: {type(colony_img)}") print(f"xr_colony type: {type(p_colony_img)}")

![image](https://github.com/Max-ChenFei/VPE_IP/assets/8528052/553f0495-1a4d-4166-92f0-b81bf17a3091)

- And we apply a `op` filter to this image by `ij.op().filter()` 
- 

HyperSphereShape = jimport('net.imglib2.algorithm.neighborhood.HyperSphereShape') radius = HyperSphereShape(2) ij.op().help("filter.variance")

apply filter

result = ij.dataset().create(colony_img) ij.op().filter().mean(result, colony_img, radius) ij.py.show(result,cmap='gray')

![image](https://github.com/Max-ChenFei/VPE_IP/assets/8528052/17b2daab-e236-49e4-ac66-d87124d7c548)

- Then we use `otsu thresholding` segmentation on it :

thresholded = ij.op().run("threshold.otsu", result) ij.py.show(thresholded,cmap='gray')

![image](https://github.com/Max-ChenFei/VPE_IP/assets/8528052/d71405eb-649d-4502-b006-4367b5167f17)

[xp1632]

Dims order in image conversion:

#64

-As we stated in Issue 64, different dim orders of different image type would be automatically reordered, base on different situation or if dim_order is specified

[xp1632]

8. Slicing and cropping java/python images

---

Here are some quick examples using the test_timeseries.tif 4D dataset with dimensions (X, Y, Channel, Time):

Syntax	Result	Shape
image	the image without any slicing	(250, 250, 3, 15)
image[:, :, 0, :]	channel 0 (blue - nucleus)	(250, 250, 15)
image[:, :, -1, :]	last channel (green - viral proteins)	(250, 250, 15)
image[100:200, 100:200,: ,:]	cropped 100 x 100 4D image, centered	(100, 100, 3, 15)

---

• Cropping Images will change the RAI type to IntervalView

• here's an example of how the java image with type RAI changes to IntervalView:

• we can wrap it as ImgPlus by ij.py.to_img() or Dataset by ij.py.to_dataset, then display it with ij.py.show():

[xp1632]

Create Numpy array of zeros in images shape

• via ij.py.initialize_numpy_image(), we can create a single image argument and returns a NumPy array of zeros in the same shape as the input image.
• This method is also used internally when we send/recieve data with ij.py.to_java() and ij.py.from_java()

[xp1632]

Run ImageJ1 marcos by ij.py.run_macro:

• To run marcos as IJ1 style, we need:
• macro code in string in certain format
• arguments of required for this macro

macro = """
#@ String name
#@ int age
#@ String city
#@output Object greeting
greeting = "Hello " + name + ". You are " + age + " years old, and live in " + city + "."
"""
args = {
    'name': 'Chuckles',
    'age': 26,
    'city': 'Saarbruecken'
}
result = ij.py.run_macro(macro, args)
print(result.getOutput('greeting'))

[xp1632]

Run ImageJ Scripts by ij.py.run_script:

• To run Scripts in ImageJ with different languages:
• We need:
• specified languages such as py, ijm, js (python, IJ1 Marco, javascript)
• scripts in string
• output variables

---

• Here's an example Groovy script to compute numerical statistics of an image:

compute_stats_script = """
#@ OpService ops
#@ net.imglib2.RandomAccessibleInterval image
#@output stats

statNames = new org.scijava.table.GenericColumn("Statistic")
statValues = new org.scijava.table.DoubleColumn("Value")
addRow = (n, v) -> { statNames.add(n); statValues.add(v.getRealDouble()) }

addRow("geometricMean", ops.stats().geometricMean(image))
addRow("harmonicMean", ops.stats().harmonicMean(image))
addRow("kurtosis", ops.stats().kurtosis(image))
addRow("max", ops.stats().max(image))
addRow("mean", ops.stats().mean(image))
addRow("median", ops.stats().median(image))
addRow("min", ops.stats().min(image))
addRow("moment1AboutMean", ops.stats().moment1AboutMean(image))
addRow("moment2AboutMean", ops.stats().moment2AboutMean(image))
addRow("moment3AboutMean", ops.stats().moment3AboutMean(image))
addRow("moment4AboutMean", ops.stats().moment4AboutMean(image))
addRow("size", ops.stats().size(image))
addRow("skewness", ops.stats().skewness(image))
addRow("stdDev", ops.stats().stdDev(image))
addRow("sum", ops.stats().sum(image))
addRow("sumOfInverses", ops.stats().sumOfInverses(image))
addRow("sumOfLogs", ops.stats().sumOfLogs(image))
addRow("sumOfSquares", ops.stats().sumOfSquares(image))
addRow("variance", ops.stats().variance(image))

stats = new org.scijava.table.DefaultGenericTable()
stats.add(statNames)
stats.add(statValues)
"""

# load a sample image
image = ij.io().open('sample-data/test_image.tif')
args = {"image":image}
result = ij.py.run_script("Groovy", compute_stats_script, args)

# convert SciJava Table to pandas DataFrame
df = ij.py.from_java(result.getOutput("stats"))
print(df)

[xp1632]

Run ImageJ Ops (including both examples from ImageJ and PyImageJ)

• Ori ImageJ Ops example: https://github.com/imagej/tutorials/blob/master/notebooks/1-Using-ImageJ/2-ImageJ-Ops.ipynb

• PyImageJ : https://github.com/imagej/pyimagej/blob/main/doc/10-Using-ImageJ-Ops.ipynb

• Reproducibility. Ops are deterministic: calling the same op twice with the same arguments yields the same result, always.

---

- Two ways of calling Op:

• 

• We'll consider using ij.op().run("filter.tubeness", ij.py.jargs(image, sigma, calibration)) this way, because it's easier to pass the parameter in a unified way.

---

- Hint for parameters

• We can get basic Op parameter information with ij.op().help()

• We can left the argument with ? suffix off because they are optional, but we should take care!!

• The parameters are left off in right-to-left order, we can pass null to hold place

---

• Gaussian op example:

---

- We should pay attention that:

1. Some parameters for Op, we need to convert it to java format by PyImageJ's ij.py.jargs()

---

Besides converting python parameters via Jargs(), calling op in ImageJ and PyImageJ are the same

here's a quick op code snippet:

import imagej

# initialize ImageJ
ij = imagej.init()
print(f"ImageJ version: {ij.getVersion()}")
[op for op in ij.op().ops() if "tube" in op]
ij.op().help("filter.tubeness")
# run tubeness op with ij.op().run()
def tubeness_1(image, sigma, calibration=[]):
    return ij.op().run("filter.tubeness", ij.py.jargs(image, sigma, calibration))

image_1 = ij.io().open('sample-data/test_still.tif')
ij.py.show(tubeness_1(image_1, sigma=10))