fractal-analytics-platform / fractal-tasks-core

Main tasks for the Fractal analytics platform
https://fractal-analytics-platform.github.io/fractal-tasks-core/
BSD 3-Clause "New" or "Revised" License
14 stars 6 forks source link

Add checks on ROI origin #530

Closed tcompa closed 11 months ago

tcompa commented 11 months ago

As of #524 (and version 0.12.0), we won't be able to use old ome-zarr's, since their FOV/well ROI tables do not have a zero origin. To avoid confusion, we should introduce two additional checks:

  1. In convert_ROI_table_to_indices, we should always check that all indices are >= 0, and fail (with a reasonable error message) otherwise.
  2. We should add a new function (something like check_ROI_table_is_supported) that gets called right after convert_ROI_table_to_indices, and also takes the ROI-table name as an input (together with the indices produced by the previous function). This should reproduce the old heuristics:

    • if name in ["FOV_ROI_table", "well_ROI_table"], then check that there is one ROI starting at (0,0,0). If not, raise an informative error.
    • else, do nothing

    The goal is to "force" users who use tasks 0.12.0 to also re-generate their OME-Zarrs with this version, so that they will have the new version of the ROIs. After a while, we should deprecate this function and remove it (TO DO: open corresponding issue).

tcompa commented 11 months ago

Both points are now implemented in #534.

For point 2, I introduced check_valid_ROI_indices(list_indices, ROI_table_name), to be called on the output of convert_ROI_to_indices (and with the appropriate table name). This function should be considered a workaround, and deprecated in a future releases (as soon as we think that OME-Zarrs created with v0.11 are not being used any more).

The only task where I did not include this check is apply_registration_to_image.py, since the ROI-indices conversion takes place within write_registered_zarr, which is not meant to use raw FOV/well ROIs (which would mean setting e.g. `registered_roi_table="FOV_ROI_table").

tcompa commented 11 months ago

After preparing a test workflow like:

  1. create-ome-zarr with version 0.11.0
  2. yokogawa-to-zarr with version 0.12.0

This is how the error appears:

TASK ERROR:Task id: 20 (Convert Yokogawa to OME-Zarr), e.workflow_task_order=1
TRACEBACK:
2023-09-28 09:01:26,305; INFO; START yokogawa_to_ome_zarr task
2023-09-28 09:01:26,318; INFO; NGFF image has num_levels=5
2023-09-28 09:01:26,318; INFO; NGFF image has coarsening_xy=2
2023-09-28 09:01:26,318; INFO; NGFF image has full-res pixel sizes [1.0, 0.1625, 0.1625]
Traceback (most recent call last):
  File "/home/tommaso/Fractal/fractal-demos/examples/server/FRACTAL_TASKS_DIR/.fractal/fractal-tasks-core0.12.0a0/venv/lib/python3.10/site-packages/fractal_tasks_core/tasks/yokogawa_to_ome_zarr.py", line 271, in <module>
    run_fractal_task(
  File "/home/tommaso/Fractal/fractal-demos/examples/server/FRACTAL_TASKS_DIR/.fractal/fractal-tasks-core0.12.0a0/venv/lib/python3.10/site-packages/fractal_tasks_core/tasks/_utils.py", line 79, in run_fractal_task
    metadata_update = task_function(**pars)
  File "pydantic/decorator.py", line 40, in pydantic.decorator.validate_arguments.validate.wrapper_function
    from contextlib import _GeneratorContextManager
  File "pydantic/decorator.py", line 134, in pydantic.decorator.ValidatedFunction.call

  File "pydantic/decorator.py", line 206, in pydantic.decorator.ValidatedFunction.execute

  File "/home/tommaso/Fractal/fractal-demos/examples/server/FRACTAL_TASKS_DIR/.fractal/fractal-tasks-core0.12.0a0/venv/lib/python3.10/site-packages/fractal_tasks_core/tasks/yokogawa_to_ome_zarr.py", line 156, in yokogawa_to_ome_zarr
    fov_indices = convert_ROI_table_to_indices(
  File "/home/tommaso/Fractal/fractal-demos/examples/server/FRACTAL_TASKS_DIR/.fractal/fractal-tasks-core0.12.0a0/venv/lib/python3.10/site-packages/fractal_tasks_core/lib_regions_of_interest.py", line 286, in convert_ROI_table_to_indices
    raise ValueError(
ValueError: ROI FOV_1 converted into negative array indices.
ZYX position: 0.0, -1517.699951171875, -1448.300048828125
ZYX pixel sizes: 1.0, 0.1625, 0.1625 (level=0)
Hint: As of fractal-tasks-core v0.12, FOV/well ROI tables with non-zero origins (e.g. the ones created with v0.11) are not supported.

Screenshot from 2023-09-28 09-02-48