ongoing efforts

[tcompa]

• [x] Moved tasks to API “V2” => 3 types (0,1,N output images) to 2 types (0+N or 1).

• [x] Unified handling of no-image-input and multi-image-input

• [x] Basic examples for new_zarr+MIP and new_zarr+copy_data

• [x] Important warning: N-dimensional parallelization (vs one-dimensional parallelization across images) is immediately risky re: huge number of parallelization items. [...]

• [x] New example: custom init task (illumination correction parallelized over channels)

• [x] New example: custom init task for registration, which pairs a reference cycle with all other cycles of each well. Note: well-handling is part of the task.

• [x] Formalized a first version of the task-output API, via TaskOutput model

• [x] Deduplicate redundant new-images list coming from subset parallelization.

• [x] Example of root_dir vs absolute path - ref #2

• [x] Source-to-output mapping: Propagating image attributes in parallel tasks (without the task needing to handle it)

• [ ] Remove default values for filters, and only consider those that are set explicitly. EDIT This is currently covered, by defaulting to None. A further review is needed.

• [x] TO CONSIDER: using both include&exclude filters, e.g. to enable selecting "images that do not have a certain attribute equal to something". EDIT: this is currently postponed, as the main intended use case (3D/2D/3D) is nicely covered by the dim=3,dim=2,dim=3 pattern (that is, using a string flag rather than a boolean attribute)

• [ ] Review and clean up the parallel-task-output merge strategies

• [x] Add safety check: Total number of custom parallel items is limited (we can later enable a safe=False override)

• [ ] Add safety check: no unintended use case is reached when applying both filters and parallelization, if there was an init task. Let’s start with very explicit if/else and error messages, and then iterate.

• [x] Fully enable 3D->2D->3D workflow example.

---

• [ ] TODO: take decision on root_dir vs absolute paths. At the moment none of the two options is blocking. The absolute-paths option requires a bit more string parsing, but that's OK (and part of it can be easily extracted into utils functions, like

  def extract_base_parts_from_list_of_absolute_paths(absolute_image_paths: listr[str]) -> dict[str, Any]:
  ...
  return dict(absolute_root_dir=absolute_root_dir, plate_names=plate_names)

• [ ] Review strategy for backwards-compatibility

[jluethi]

Discussion on "Remove default values for filters, and only consider those that are set explicitly": (path = "plate.zarr/B/03/0") (path = "plate.zarr/B/03/0_corr", raw=False)

Still have ability to load raw=False and only return the second item

[jluethi]

Some validation of tasks would be very interesting. For example: MIP requires images with "Dimensionality = 3D"

[tcompa]

To evaluate: Should parallel tasks return something like new_image and/or edited_image, rather than new_images and/or edited_images? TBD

[tcompa]

To evaluate: How can an image be removed from dataset.images? First proposal: tasks don't remove images, but the fractal-server API makes it easy to play with dataset.images if needed Second proposal: next to new_images and edited_images, a task can also output removed_images.

TBD