Possible 【Auto Op generation enhancement】

[xp1632]

• After some investigation, for deeper enhancement of Auto generation pipeline:

• 1. Auto for IJ1 Plugin is too hard since IJ1 plugin doesn't provide, (get from @input in plugin's template somewhere?

• 2. Auto for 'user' without inputting 'code snippet', relatively easier, cost losing robustness of Op calling

• 3. Auto workflow conversion, normal hard but could be feasible, is this useful?

---

• [x] - Not so deep enhancement 1 that should be added:

---

• We checked the code snippet for calling threshold.otsu again
• and found the different of calling op is caused by link node between actual calling
• we have a unified way which is the original rule ij.op().run("op_name", ij.py.jargs( ** Input parameters **))
• More details in https://github.com/Max-ChenFei/VPE_IP/issues/80#issuecomment-2035158298

---

• Now we extract op parameters in Pipeline 5.1 by

  # Extract the arguments of 'ij.py.jargs'
  self.jargs_args.extend([astor.to_source(arg).strip() for arg in node.args[1].args])

  -because by then we are using filter.median as example:

-result_image= ij.op().run(Op_name, ij.py.jargs(result_image, process_image, HyperSphereShape(range),None))

• it requires more input arguments other than just image

• However, we also have other Ops that is easier to use such as:

mask = ij.op().run("threshold.otsu", smoothed)

• We need to add this into the ast extraction rule too

• [x] Not so deep enhancement 2 that should be checked:

---

• We checked again and found it doesn't have type conversion problem between python and java

• More details could be found in https://github.com/Max-ChenFei/VPE_IP/issues/80#issuecomment-2027290547

• By checking the JIPipe source code again, we found it also used cInfo():

• which is CommandInfo :https://javadoc.scijava.org/SciJava/org/scijava/command/CommandInfo.html?is-external=true

• for Command Module of SciJava https://javadoc.scijava.org/SciJava/org/scijava/command/CommandModule.html

• I am not sure if this would provide us more or more accurate information than OpInfo : https://javadoc.scijava.org/ImageJ2/net/imagej/ops/OpInfo.html

• we'll see after seminar

[xp1632]

VP node for Auto Op Generation Node should look like :

1. We create a large string area for displaying the user code snippet as in the picture left, and add an output handle to it

2. Or, we make the path string input area in the picture right really large

[xp1632]

3. Auto workflow conversion

• We did some preparation attempt for this method:

• As in https://jckling.github.io/2021/07/14/Other/Python%20ast%20%E6%A8%A1%E5%9D%97%E4%BD%BF%E7%94%A8/index.html

• 

• We could get the execution order and operation dependency based on past

• Here's a simpler illstrution of ast: https://kamneemaran45.medium.com/python-ast-5789a1b60300

---

• By This version of code
• It is possible to get the execution order, and whose input is whose output information
• However, the reassignment of a variable with same name is not solved yet:

import ast

code = """
HyperSphereShape = sj.jimport("net.imglib2.algorithm.neighborhood.HyperSphereShape")
smoothed = ij.op().run("create.img", jslice)
smoothed = ij.op().run("filter.median", ij.py.jargs(smoothed, cells_slice, HyperSphereShape(5)))
mask = ij.op().run("threshold.otsu", smoothed)
mask = ij.op().run("morphology.dilate", mask, HyperSphereShape(1))
mask = ij.op().run("morphology.fillHoles", mask)
ij.py.show(mask)
"""

tree = ast.parse(code)

# List to store the operations and their output variables
ops = []

for node in ast.walk(tree):
    if isinstance(node, ast.Assign):
        # If the node is an assignment, get the variable name and the operation that produces it
        var_name = node.targets[0].id
        if isinstance(node.value, ast.Call):
            func = node.value.func
            if isinstance(func, ast.Attribute) and func.attr == 'run':
                op_name = node.value.args[0].s
                ops.append((op_name, var_name))  # Add the operation and its output variable to the list
                print(f"Variable '{var_name}' is produced by operation '{op_name}'")

    elif isinstance(node, ast.Call):
        func = node.func
        if isinstance(func, ast.Attribute) and func.attr == 'run':
            # If the node is a function call, check if any of the arguments are variables produced by previous operations
            for arg in node.args[1:]:
                if isinstance(arg, ast.Name):
                    for i in reversed(range(len(ops))):
                        if ops[i][1] == arg.id:
                            print(f"Variable '{arg.id}' produced by operation '{ops[i][0]}' is used as an argument in operation '{node.args[0].s}'")
                            break

---

• More details of challenges for this method is on picture in the beginning

[xp1632]

Answer to Fei's HypersphereShape(5) handle type passing value question

• Currently, for java datatype Hypersphere, we use a mapping table between java type and python type

• And simply map Hypersphere to integer datatype that is supported in Chaldene

• What if in certain case, we need to connect this (Hypersphere handle) with another Hypersphere handle?

---

• The answer is still simple, we keep one more information item in this handle's specification
• which is java_type: Hypersphere
• And we still pass the integer value 5, between these two handles
• but we'll also import the java type we need by scyjava's jimport, and we use it as Hypersphere(5)

---

And the two node handles can only be connected if both datatype and java_type match

[xp1632]

Next step for auto generation pipeline

• After a short discussion with Fei, he mentioned batch generation
• does auto = batch?
• I doubt that.

---

• Also, I don't understand how we can do batch for a whole library or something

  ---

• But one thing I know, is that:

• I need to first fix the only recognizing jarg() parameters and also add only op() recognition in the pipeline

• then let's read the JIPipe code again and see how they deal with the code template thing

• Lastly, I am worried about the IJ1 plugins that don't provide input/output information interface

[xp1632]

Where to iterate all ImageJ2 Ops?

In OpService