First VP ImageJ node example_and_solutions for problems

[xp1632]

First VP ImageJ node example:

• We think about the details of converting ImageJ functions --> VP node via PyImageJ:
• And here are the current solutions:

---

1. What code should be in the VP import part ?

   • We have

     • Python normal import: to import python libraries
     • Java Jimport: to import java libraries

       HyperSphereShape = jimport('net.imglib2.algorithm.neighborhood.HyperSphereShape')
       radius = HyperSphereShape(2)

     • ImageJ initialization to initialize PyImageJ and ImageJ in the beginning, this part is necessary for all ImageJ functions:

       
       import imagej

     initialize ImageJ

     ij = imagej.init('sc.fiji:fiji:2.14.0') print(f"ImageJ version: {ij.getVersion()}")

---

Current Solution

• VP import part for ImageJ should be :
  • Python normal import
  • and ImageJ initialization
• Java JImport should be in the VP code part

---

Reasons

• VP import is stored as Set to avoid duplication

• VP code is stored as string which can not avoid duplication

• ImageJ Initialization is required at the beginning of the code before running any ImageJ functions, so for every function ImageJ node, we'll have the ImageJ Initialization part.

• We don't want to connect to ImageJ gate multiple times, so ImageJ Initialization should be not be duplicated.

• Thus we make the decision of putting the ImageJ Initialization part into VP import part

---

Small concerns

1. Orders of VP import:

• While we decided to put ImageJ Initialization part in the VP import
• we don't know where this line of code ij = imagej.init('sc.fiji:fiji:2.14.0') would locate
• It could locate in the middle of the rest Python normal imports, would it still work?
• Yes

[xp1632]

2. We want to avoid dealing with Knowledge Graph and Display nodes in the first stage, what contents should be in the first VP ImageJ workflow?

---

• 2.1 For Display nodes, now we don't have a formal Display nodes in VP project.
  • we have captureImageCode and captureImageFunctions in python generator to show intermediate image result of a node on its node panel
  • For now we can just print the result in the following Jupyter cell by PyImageJ's ij.py.show(java image object) as a temporary measure
  • And this ij.py.show(java image object) should be added within every VP node that has image invovled

---

• 2.2 For Converters in Knowledge Graph, we consider the KG part as the next step, we do not want to include imageJ converters in current knowledge graph.

• Thus we'll unify the input/output image Type as numpy.ndarray

• And do the conversion inside of VP function node via PyImageJ's from_java(), to_java()

• Also, in PyImageJ only IJ1 ImagePlus and RandomAccessibleInterval are converted to numpy, so we'll use ij.IJ.openImage() to read the image as the correct form

• Images we get from ij.io().open() returns ImageJ2 Dataset java object.
• Image we get from ij.IJ.openImage() legacy opener returns ImageJ1 ImagePlusjava object

[xp1632]

Here's the draft of our first VP node workflow:

[xp1632]

Ori_VP_ReadImage_Node

• JSON file:

{
  "description": "Image read and write functions from torchvision.io",
  "enable": true,
  "nodes": {
    "read_image": {
      "type": "read_image",
      "category": "function",
      "title": "read_image",
      "tooltip": "Reads a JPEG or PNG image into a 3 dimensional RGB or grayscale Tensor. Optionally converts the image to the desired format. The values of the output tensor are uint8 in [0, 255].",
      "externalImports": "from torchvision import io\nfrom torchvision.io import ImageReadMode",
      "codeGenerator": "function code(inputs, outputs, node, generator) {\n    if (inputs[2] === 'ImageReadMode.RGB')\n        return `${outputs[1]} = io.read_image(${inputs[1]}, ${inputs[2]})\\n${outputs[1]} = {\\n  'value': ${outputs[1]},\\n  'dataType': 'torch.tensor',\\n  'metadata': {\\n    'colorChannel': 'rgb',\\n    'channelOrder': 'channelFirst',\\n    'isMiniBatched': False,\\n    'intensityRange': '0-255',\\n    'device': 'cpu'\\n  }\\n}\\n${outputs[0]}`;\n    if (inputs[2] === 'ImageReadMode.GRAY')\n        return `${outputs[1]} = {\\n  'value': ${outputs[1]},\\n  'dataType': 'torch.tensor',\\n  'metadata': {\\n    'colorChannel': 'grayscale',\\n    'channelOrder': 'channelFirst',\\n    'isMiniBatched': False,\\n    'intensityRange': '0-255',\\n    'device': 'cpu'\\n  }\\n}\\n${outputs[0]}`;\n}",
      "inputs": {
        "execIn": {
          "title": "execIn",
          "tooltip": "execIn",
          "dataType": "exec",
          "showWidget": false,
          "showTitle": false
        },
        "path": {
          "title": "path",
          "dataType": "string",
          "tooltip": "path(str) - path of the JPEG or PNG image."
        },
        "mode": {
          "title": "mode",
          "dataType": "imageio.ImageReadMode",
          "defaultValue": "ImageReadMode.RGB",
          "tooltip": "mode(ImageReadMode) - The read mode used for optionally converting the image. Default: ImageReadMode.UNCHANGED."
        }
      },
      "outputs": {
        "execOut": {
          "title": "execOut",
          "tooltip": "execOut",
          "dataType": "exec",
          "showWidget": false,
          "showTitle": false
        },
        "image": {
          "title": "image",
          "dataType": "image",
          "defaultValue": {
            "dataType": "torch.tensor"
          },
          "tooltip": "{dataType: torch.tensor, value, layout: [chw], colorMode: [rgb, grayscale], intensityRange: 0-255' device: cpu}"
        }
      }
    }
  }
}

[xp1632]

VP_IJ_read_image_node is working:

IJ_read_image6.json

• we first determine the code snippet that we want the node to provide :

• then we add ${} syntax on it :

• 

• code generator:

  function code(inputs, outputs, node, generator) {
      return `${outputs[1]} = ij.py.initialize_numpy_image(ij.io().open(${inputs[1]}))\n${outputs[1]} = ij.py.rai_to_numpy(ij.io().open(${inputs[1]}),${outputs[1]})\nij.py.show(${outputs[1]})\n${outputs[1]} = {\n  'value': ${outputs[1]},\n  'dataType': 'torch.tensor',\n  'metadata': {\n    'colorChannel': 'rgb',\n    'channelOrder': 'channelFirst',\n    'isMiniBatched': False,\n    'intensityRange': '0-255',\n    'device': 'cpu'\n  }\n}\n${outputs[0]}`;
  }

[xp1632]

Problems with Intermediate Image Display on Node:

• The image we read is in tif format which supported by ImageJ, not by original pytorch library
• I have already converted the image type to numpy.ndarray, a image type that should be supported in VP
• but we met problems as :

• 

---

• At first glance, the problems are caused by different dimension orders we expect in numpy and the numpy we get from PyImageJ

---

• Solution: we write the metadata for numpy wrong:

we wrote :

n_1_image = {
  'value': n_1_image,
  'dataType': 'torch.tensor',
  'metadata': {
    'colorChannel': 'rgb',
    'channelOrder': 'channelFirst',
    'isMiniBatched': False,
    'intensityRange': '0-255',
    'device': 'cpu'
  }
}

• it should be

n_1_image = {
  'value': n_1_image,
  'dataType': '**numpy.ndarray**',
  'metadata': {
    'colorChannel': 'rgb',
    'channelOrder': '**channelLast**',
    'isMiniBatched': False,
    'intensityRange': '0-255',
    'device': 'cpu'
  }
}

[xp1632]

CaptureImageCode structure:

• We still met some problems of capturing the intermediate image functions
• So we check the CaptureImageCode structure:

• 

---

And find out the problems should happen in two parts:

1. The conversion part:

• The start conversion type is numpy.ndarray, the destination type is current

• we should know the RGB numpy.ndarray has three dimension such as the following jpg image (1500,1063,3)

• however the single-channel tif image we read by ij.io().open() has shape as (250,250), we don't have the third dimension

---

2. The PIL image part:

• In CaptureImageCode function returns a code snippet that contains VP import, captureImageFunction, commInJupyter , convertFunctions

• we get a pillow image(PIL) from function Image.fromarray()

• This functions also requires numpy array input

[xp1632]

Solution for the different required numpy shape:

• before we think this image is a RGB image, because it has color in it

• 

---

• However, it's a grayscale image with a colormapping in matlibplot
• It explains why we get (250,250) as its shape, and the third colorchannel is missing, because there's no color channel for grayscale image
• We are misleaded by the color on a grayscale picture, but actually the color comes from colormapping in matlibplot

https://matplotlib.org/stable/gallery/color/colormap_reference.html

---

The picture without color mapping should look like:

• and of course it has shape of (250,250) when it is a !!!!! grayscale image

[xp1632]

CaptureImageCode inside problems caused by non- unique dom id

are in Issue https://github.com/Max-ChenFei/VPE_IP/issues/70

• after solving that issue, we find out the image is successfully transferred to node panel via jupyter communication channel
• but it doesn't looks correct

---

---

• is it caused by

• [x] wrong PIL conversion?

• [ ] wrong KG conversion?

  • [ ] wrong metadata?

• [ ] wrong encode and decode?

---

• We displayed the intermediate state of image to find which process went wrong:

[xp1632]

The PIL conversion looks wrong, we test some cases to see which part went wrong:

1. We'll test jpg image read by ori read_image_io's case's PIL image

• The PIL image is correctly displayed.

---

• Which means convert jpg image to PIL image should not change how it looks

---

2. So it means, while this image and our image are both converted from array to PIL image, it's dtype or shape or range could be different

• The image that is successfully converted from array to PIL image without changing is unit8 dtype with range 0,255

---

2.1 Our Java object tif image we get from PyImage, then converted to numpy array has different type and range:

PIL images expect pixel values to be in the range of 0-255 for 8-bit images. If the numpy image has a different data type or the pixel values are not in the expected range, the resulting PIL image may not look correct.

---

• So this is the reason that our tif image could not be correctly displayed after Image.fromArray() array ->PIL conversion

- Solution

• we normalized the array into range [0,255] and dtype uint8 by: np_img_1707918575145['value'] = ((np_img_1707918575145['value'] - np_img_1707918575145['value'].min()) / (np_img_1707918575145['value'].max() - np_img_1707918575145['value'].min()) * 255).astype('uint8')

• then the image is correctly displayed after PIL conversion:

---

Conclusion for CaptureImaegCode functions:

1. we should only keep the desired node panel as Watched and unwatch all the rest node in other cells, notebooks, in the whole jupyter environment to keep its unique dom_id

   • otherwise, the image would be pass to the first VP cell with first watch node

2. we should normalize the image array into range [0,225] and type uint8 with code like
   np_img_1707918575145['value'] = ((np_img_1707918575145['value'] - np_img_1707918575145['value'].min()) / (np_img_1707918575145['value'].max() - np_img_1707918575145['value'].min()) * 255).astype('uint8') so it could be correctly displayed

   • And I was wondering, should we keep this normalization part in every node?
   • Or in CaptureImageFunction before its PIL conversion Image.fromArray
   • Now I don't want to change CaptureImageFunction itself so I'll just normalize the output numpy array of our IJ_VP_Node

[xp1632]

- Working node:

IJ_read_image32.json

• the grayscale image is default color mapped by Matplotlib in the next cell

[xp1632]

Proceed to Op filters

• Now we know how to make the code and the display work
• we can take this IJ_VP_Node as a start/template, and add one Op to process the image

• Then we choose a more complex example workflow and implement it

---

• we'll have a IJ_Mean_Filter_Node that has input image, radius
• And show result with gray cmpa
• we'll have another IJ_otsu_segmentation node

-I think for segmention, this image is a better example: https://imagej.net/plugins/auto-threshold#available-methods

• Here we can refer to Jipype, to see if we could call all the different segmentations in one Node with different option in node panel

---

- We could also print some short introductory messages before each image display

• To show the state of image step by step

[xp1632]

Modification of IJ_Read Node before moving on to Op

We want the current IJ_read_image node to

---

1. show grayscale image without matplotlib's default color mapping

   • only set cmap=gray for grayscale image with 2D shape(x,y)

2. show color tif image and RGB jpg image as it is(with shape x,y,channel)

   • in current version of IJ_read_image, when reading jpg image, we get numpy array with shape (channel, x, y), channel is in the first order of dimension
   • however in current knowledge graph, numpy array are ChannelLast
   • we need to transpose the array, for example, from (3, x, y) to (x,y,3)

3. show helping messages for time-series with shape (x, y, channel, time)

   • only 2d images can be displayed on jupyter, so we'll recommend the user to slice the time series image and then display it

4. show introductory messages before every image display in jupyter:

   • for example : Original Image:, After Mean Filter, After Segmentation
   • If we want to show more information of current image, we can also display the dump info after reading the image

     
     def dump_info(image):
     """A handy function to print details of an image object."""
     name = image.name if hasattr(image, 'name') else None # xarray
     if name is None and hasattr(image, 'getName'): name = image.getName() # Dataset
     if name is None and hasattr(image, 'getTitle'): name = image.getTitle() # ImagePlus
     print(f" name: {name or 'N/A'}")
     print(f" type: {type(image)}")
     print(f"dtype: {image.dtype if hasattr(image, 'dtype') else 'N/A'}")
     print(f"shape: {image.shape}")
     print(f" dims: {image.dims if hasattr(image, 'dims') else 'N/A'}")

dump_info(dataset)

**
![image](https://github.com/Max-ChenFei/VPE_IP/assets/8528052/cfd18293-5a8d-413f-aa30-3c47899eb4fc)
**
---

**We would like our `IJ_read_image` to work as follows**

![image](https://github.com/Max-ChenFei/VPE_IP/assets/8528052/e18fb088-3b82-4b7c-9841-2a7d2f549915)

[xp1632]

Final Code Snippet for IJ_read_image:

import imagej
ij = imagej.init('sc.fiji:fiji:2.14.0')
import numpy as np
tif_image = ij.py.initialize_numpy_image(ij.io().open('retina.jpg'))
n_1_image = ij.py.rai_to_numpy(ij.io().open('retina.jpg'),tif_image)
number_of_dims = np.ndim(n_1_image)
def image_info(image):
    # A handy function to print details of an image object.
    print("--- Image Information ---")
    name = image.name if hasattr(image, 'name') else None # xarray
    if name is None and hasattr(image, 'getName'): name = image.getName() # Dataset
    if name is None and hasattr(image, 'getTitle'): name = image.getTitle() # ImagePlus
    print(f" type: {type(image)}")
    print(f"dtype: {image.dtype if hasattr(image, 'dtype') else 'N/A'}")
    print(f"shape: {image.shape}")
    print(f" dims: {image.dims if hasattr(image, 'dims') else 'N/A'}")
    print ("-------------------------")
image_info(n_1_image);
if number_of_dims == 2:
    #Grayscale image
    print("Grayscale image:")
    ij.py.show(n_1_image,cmap='gray')
if number_of_dims == 3:
    #RGB image 
    print("Input Image:")
    #transpose (channel,x,y) -> (x,y,channel)
    n_1_image = np.transpose(n_1_image, (1, 2, 0))
    print(n_1_image.shape)
    ij.py.show(n_1_image)
if number_of_dims == 4:
    #time series image
    print("Currently we focus on 2d images, please slice the 3d image for displaying in jupyter")
#Normalization to range (0,255), uint8 
n_1_image = ((n_1_image - n_1_image.min()) / (n_1_image.max() - n_1_image.min())* 255).astype('uint8')
color_channel = 'grayscale' if number_of_dims == 2 else 'rgb'
n_1_image = {
    'value': n_1_image,
    'dataType': 'numpy.ndarray',
    'metadata': {
    'colorChannel': color_channel,
    'channelOrder': 'channelLast',
    'isMiniBatched': False,
    'intensityRange': '0-255',
    'device': 'cpu'
    }
}

---

• we tested the code with color tif, jpg, grayscale tif, 4d timeseries tif

1. Display Color tif:

2. Display Grayscale tif without color mapping:

3. Display 4d timeseries tif

• currently we focus on processing and displaying 2d images
• so we give a hint for the user to slice 3d images

[xp1632]

Final IJ_Read_Image Code --> Node

• We need to consider the enum option in node panel for colorChannel
• Although we don't need that in the input, we need it to be correct in metadata for further image processing and type conversion

---

• We don't want to add \n to each line of code and replace the input and output manually
• so we have a json_node_generator.py to format the code we need:

format_code() in json_node_generator

def format_code(code, input1, output1):
    # Replace the input and output names in the code
    code = code.replace("'{}'".format(input1), "${inputs[1]}")
    code = code.replace(output1, "${outputs[1]}")

    # Add a visible '\n' at the end of each line
    code = '\\n'.join(code.strip().splitlines()) + '\\n'
    code += '${outputs[0]}'
    return code

# Test the function
code = """
tif_image = ij.py.initialize_numpy_image(ij.io().open('retina.jpg'))
n_1_image = ij.py.rai_to_numpy(ij.io().open('retina.jpg'),tif_image)
n_1_image = ((n_1_image - n_1_image.min()) / (n_1_image.max() - n_1_image.min())* 255).astype('uint8')
ij.py.show(n_1_image)
n_1_image = {
    'value': n_1_image,
    'dataType': 'numpy.ndarray',
    'metadata': {
        'colorChannel': 'grayscale',
        'channelOrder': 'channelLast',
        'isMiniBatched': False,
        'intensityRange': '0-255',
        'device': 'cpu'
    }
}
"""
formatted_code = format_code(code, "retina.jpg", "n_1_image")
print(formatted_code)

---

• The output is

tif_image = ij.py.initialize_numpy_image(ij.io().open(${inputs[1]}))\n${outputs[1]} = ij.py.rai_to_numpy(ij.io().open(${inputs[1]}),tif_image)\n${outputs[1]} = ((${outputs[1]} - ${outputs[1]}.min()) / (${outputs[1]}.max() - ${outputs[1]}.min())* 255).astype('uint8')\nij.py.show(${outputs[1]})\n${outputs[1]} = {\n    'value': ${outputs[1]},\n    'dataType': 'numpy.ndarray',\n    'metadata': {\n        'colorChannel': 'grayscale',\n        'channelOrder': 'channelLast',\n        'isMiniBatched': False,\n        'intensityRange': '0-255',\n        'device': 'cpu'\n    }\n}\n${outputs[0]}

• we'll test if it works in VP_IJ_node, if so, we'll use this to generate the return of codeGenerator

---

Thoughts for IJ_Op_VP_node template and auto-generation only for this part

• This format_code() only generator part of JSON
• After we have a template for IJ2 Op plugin, IJ1 Marco plugin , is it possible that we make an auto_Op_Node_generator etc. based on a template?

[xp1632]

Auto-generated part of json node pass:

- Enhancement of Auto_Json_generation is in Issue https://github.com/Max-ChenFei/VPE_IP/issues/72

---

• We test more complicated code for partial auto generation:
• it passed:

Copied textual Code

import imagej
ij = imagej.init('sc.fiji:fiji:2.14.0')
import numpy as np
tif_image = ij.py.initialize_numpy_image(ij.io().open('retina.jpg'))
n_1_image = ij.py.rai_to_numpy(ij.io().open('retina.jpg'),tif_image)
number_of_dims = np.ndim(n_1_image)
def image_info(image):
    # A handy function to print details of an image object.
    print("--- Image Information ---")
    name = image.name if hasattr(image, 'name') else None # xarray
    if name is None and hasattr(image, 'getName'): name = image.getName() # Dataset
    if name is None and hasattr(image, 'getTitle'): name = image.getTitle() # ImagePlus
    print(f" type: {type(image)}")
    print(f"dtype: {image.dtype if hasattr(image, 'dtype') else 'N/A'}")
    print(f"shape: {image.shape}")
    print(f" dims: {image.dims if hasattr(image, 'dims') else 'N/A'}")
    print ("-------------------------")
image_info(n_1_image);
if number_of_dims == 2:
    #Grayscale image
    print("Grayscale image:")
    ij.py.show(n_1_image,cmap='gray')
if number_of_dims == 3:
    #RGB image 
    print("Input Image:")
    #transpose (channel,x,y) -> (x,y,channel)
    n_1_image = np.transpose(n_1_image, (1, 2, 0))
    print(n_1_image.shape)
    ij.py.show(n_1_image)
if number_of_dims == 4:
    #time series image
    print("Currently we focus on 2d images, please slice the 3d image for displaying in jupyter")
#Normalization to range (0,255), uint8 
n_1_image = ((n_1_image - n_1_image.min()) / (n_1_image.max() - n_1_image.min())* 255).astype('uint8')
color_channel = 'grayscale' if number_of_dims == 2 else 'rgb'
n_1_image = {
    'value': n_1_image,
    'dataType': 'numpy.ndarray',
    'metadata': {
    'colorChannel': color_channel,
    'channelOrder': 'channelLast',
    'isMiniBatched': False,
    'intensityRange': '0-255',
    'device': 'cpu'
    }
}

---

Partial_Json_generator:

def format_code(code, input1, output1):
    # Replace the input and output names in the code
    code = code.replace("'{}'".format(input1), "${inputs[1]}")
    code = code.replace(output1, "${outputs[1]}")

    # Add a visible '\n' at the end of each line
    code = '\\n'.join(code.strip().splitlines()) + '\\n'
    code += '${outputs[0]}'

    # Add backticks at the beginning and end of the code
    code = '`' + code + '`'
    return code
# Test real code
code = """
tif_image = ij.py.initialize_numpy_image(ij.io().open('test_grayscale_image.tif'))
n_1_image = ij.py.rai_to_numpy(ij.io().open('test_grayscale_image.tif'),tif_image)
number_of_dims = np.ndim(n_1_image)
def image_info(image):
    # A handy function to print details of an image object.
    print("--- Image Information ---")
    name = image.name if hasattr(image, 'name') else None # xarray
    if name is None and hasattr(image, 'getName'): name = image.getName() # Dataset
    if name is None and hasattr(image, 'getTitle'): name = image.getTitle() # ImagePlus
    print(f" type: {type(image)}")
    print(f"dtype: {image.dtype if hasattr(image, 'dtype') else 'N/A'}")
    print(f"shape: {image.shape}")
    print(f" dims: {image.dims if hasattr(image, 'dims') else 'N/A'}")
    print ("-------------------------")
image_info(n_1_image);
if number_of_dims == 2:
    #Grayscale image
    print("Grayscale image:")
    ij.py.show(n_1_image,cmap='gray')
if number_of_dims == 3:
    #RGB image 
    print("Input Image:")
    #transpose (channel,x,y) -> (x,y,channel)
    n_1_image = np.transpose(n_1_image, (1, 2, 0))
    print(n_1_image.shape)
    ij.py.show(n_1_image)
if number_of_dims == 4:
    #time series image
    print("Currently we focus on 2d images, please slice the 3d image for displaying in jupyter")
#Normalization to range (0,255), uint8 
n_1_image = ((n_1_image - n_1_image.min()) / (n_1_image.max() - n_1_image.min())* 255).astype('uint8')
color_channel = 'grayscale' if number_of_dims == 2 else 'rgb'
n_1_image = {
    'value': n_1_image,
    'dataType': 'numpy.ndarray',
    'metadata': {
    'colorChannel': color_channel,
    'channelOrder': 'channelLast',
    'isMiniBatched': False,
    'intensityRange': '0-255',
    'device': 'cpu'
    }
}
"""
formatted_code = format_code(code, "test_grayscale_image.tif", "n_1_image")
print(formatted_code)

---

json_generator_output:

`tif_image = ij.py.initialize_numpy_image(ij.io().open(${inputs[1]}))\n${outputs[1]} = ij.py.rai_to_numpy(ij.io().open(${inputs[1]}),tif_image)\nnumber_of_dims = np.ndim(${outputs[1]})\ndef image_info(image):\n    # A handy function to print details of an image object.\n    print("--- Image Information ---")\n    name = image.name if hasattr(image, 'name') else None # xarray\n    if name is None and hasattr(image, 'getName'): name = image.getName() # Dataset\n    if name is None and hasattr(image, 'getTitle'): name = image.getTitle() # ImagePlus\n    print(f" type: {type(image)}")\n    print(f"dtype: {image.dtype if hasattr(image, 'dtype') else 'N/A'}")\n    print(f"shape: {image.shape}")\n    print(f" dims: {image.dims if hasattr(image, 'dims') else 'N/A'}")\n    print ("-------------------------")\nimage_info(${outputs[1]});\nif number_of_dims == 2:\n    #Grayscale image\n    print("Grayscale image:")\n    ij.py.show(${outputs[1]},cmap='gray')\nif number_of_dims == 3:\n    #RGB image \n    print("Input Image:")\n    #transpose (channel,x,y) -> (x,y,channel)\n    ${outputs[1]} = np.transpose(${outputs[1]}, (1, 2, 0))\n    print(${outputs[1]}.shape)\n    ij.py.show(${outputs[1]})\nif number_of_dims == 4:\n    #time series image\n    print("Currently we focus on 2d images, please slice the 3d image for displaying in jupyter")\n#Normalization to range (0,255), uint8 \n${outputs[1]} = ((${outputs[1]} - ${outputs[1]}.min()) / (${outputs[1]}.max() - ${outputs[1]}.min())* 255).astype('uint8')\ncolor_channel = 'grayscale' if number_of_dims == 2 else 'rgb'\n${outputs[1]} = {\n    'value': ${outputs[1]},\n    'dataType': 'numpy.ndarray',\n    'metadata': {\n    'colorChannel': color_channel,\n    'channelOrder': 'channelLast',\n    'isMiniBatched': False,\n    'intensityRange': '0-255',\n    'device': 'cpu'\n    }\n}\n${outputs[0]}`

[xp1632]

Requirement for code that we want to autoformat:

1. no extra space or line break between lines ? - 2. Value of node name and node type in json object should be the same, title doesn't matter for Node but is important for user to read
   3. We support '#' for commenting, we don't support """

[xp1632]

calling Ops Informations are in https://github.com/Max-ChenFei/VPE_IP/issues/62#issuecomment-1960070119

[xp1632]

Op Segmentation workflow:

• We've finalized the showcase workflow:

1. Cell Segmentation based on existing ops (https://py.imagej.net/en/latest/Classic-Segmentation.html#segmentation-workflow-with-imagej2)

1.2 Particles Analysis: (https://py.imagej.net/en/latest/Puncta-Segmentation.html)

1.4 Frangi vesselness from a MRA brain image: (https://github.com/imagej/tutorials/blob/master/notebooks/1-Using-ImageJ/2-ImageJ-Ops.ipynb)

---

• These two section have a lot of ops and have more detailed usecase in the bottom of https://github.com/imagej/tutorials/blob/master/notebooks/1-Using-ImageJ/2-ImageJ-Ops.ipynb
• we could focus on this after finishing the sole

1.5 Thresholding, tons of thresholding op: https://github.com/imagej/tutorials/blob/master/notebooks/1-Using-ImageJ/2-ImageJ-Ops.ipynb we should use test image here: https://imagej.net/plugins/auto-threshold#available-methods

1.6 Morphology https://github.com/imagej/tutorials/blob/master/notebooks/1-Using-ImageJ/2-ImageJ-Ops.ipynb

---

(-)2. More advanced cellpose segmentation for future case: (https://py.imagej.net/en/latest/Cellpose-StarDist-Segmentation.html)

(x)3. More more advanced texture analysis and add red rectangle on image: https://py.imagej.net/en/latest/GLCM.html#using-pyimagej-and-imagej-ops-for-glcm-texture-analysis

[xp1632]

Cell Segmentation Workflow

1. 3d_image_slicing: cell_segmentation_slicing6.json

2. edge_preserve_smoothing cell_segmentation_slicing6_edge7.json

3. otsu_segmentation cell_segmentation_s6_e7_otsu1.json

4.1 morphology filter_dilation

cell_segmentation_s6_e7_o1_morph3.json

4.2 morphology filter_fillHoles cell_segmentation_s6_e7_o1_m3_fillHole1.json

5. watershed labeling

[xp1632]

Watershed Problems

• we might problems at watershed labeling:

• In short, the ImgLabeling java object is not supported in PyImageJ and it's hard for us to convert it into normal python datatype

• then we can not display it on NodePanel via PIL

• but actually it can still shown in the cell:

• 

• but the result is also not as good as it should be

• so we'll omit it first and fix it later