fractal-analytics-platform / fractal-tasks-core

Main tasks for the Fractal analytics platform
https://fractal-analytics-platform.github.io/fractal-tasks-core/
BSD 3-Clause "New" or "Revised" License
14 stars 6 forks source link

Make multiplexing converter work with arbitrarily named acquisitions #782

Closed jluethi closed 3 months ago

jluethi commented 4 months ago

We've generalized the handling of acquisition names with the 1.0.0 release of fractal-tasks-core. But some tasks still assume acquisitions are named "0", "1", "2". Let's generalize that.

First issue Nicole found:

TASK ERROR: Task name: Convert Cellvoyager Multiplexing to OME-Zarr, position in Workflow: 0
TRACEBACK:
Traceback (most recent call last):
  File "/path/to/python/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 116, in run_single_task
    raise e
  File "/path/to/python/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 105, in run_single_task
    _call_command_wrapper(
  File "/path/to/python/lib64/python3.9/site-packages/fractal_server/app/runner/v2/runner_functions_low_level.py", line 52, in _call_command_wrapper
    raise TaskExecutionError(err)
fractal_server.app.runner.exceptions.TaskExecutionError: 2024-07-08 13:07:15,771; INFO; START cellvoyager_to_ome_zarr_init_multiplex task
2024-07-08 13:07:15,772; INFO; [glob_with_multiple_patterns] patterns=['*.tif']
2024-07-08 13:07:15,800; INFO; [glob_with_multiple_patterns] Found 3512 items
2024-07-08 13:07:15,803; WARNING; Skipping "DC_DCAM#2_CAM4_BIN2.tif". Original error: Something wrong with filename='DC_DCAM#2_CAM4_BIN2', TFLAZC='BIN2'
2024-07-08 13:07:15,812; WARNING; Skipping "SC_BP676-29_60xW_M01_CH04.tif". Original error: Something wrong with filename='SC_BP676-29_60xW_M01_CH04', TFLAZC='CH04'
2024-07-08 13:07:15,823; WARNING; Skipping "SC_BP600-37_60xW_M01_CH03.tif". Original error: Something wrong with filename='SC_BP600-37_60xW_M01_CH03', TFLAZC='CH03'
2024-07-08 13:07:15,824; WARNING; Skipping "DC_DCAM#2_CAM4.tif". Original error: Something wrong with filename='DC_DCAM#2_CAM4', TFLAZC='CAM4'
2024-07-08 13:07:15,836; WARNING; Skipping "DC_DCAM#1_CAM3_BIN2.tif". Original error: Something wrong with filename='DC_DCAM#1_CAM3_BIN2', TFLAZC='BIN2'
2024-07-08 13:07:15,853; WARNING; Skipping "SC_BP445-45_60xW_M01_CH01.tif". Original error: Something wrong with filename='SC_BP445-45_60xW_M01_CH01', TFLAZC='CH01'
2024-07-08 13:07:15,856; WARNING; Skipping "SC_BP525-50_60xW_M01_CH02.tif". Original error: Something wrong with filename='SC_BP525-50_60xW_M01_CH02', TFLAZC='CH02'
2024-07-08 13:07:15,858; WARNING; Skipping "DC_DCAM#1_CAM3.tif". Original error: Something wrong with filename='DC_DCAM#1_CAM3', TFLAZC='CAM3'
Traceback (most recent call last):
  File "/path/to/FRACTAL_TASK_DIR/.fractal/fractal-tasks-core1.1.0/venv/lib/python3.9/site-packages/fractal_tasks_core/tasks/cellvoyager_to_ome_zarr_init_multiplex.py", line 521, in <module>
    run_fractal_task(
  File "/path/to/FRACTAL_TASK_DIR/.fractal/fractal-tasks-core1.1.0/venv/lib64/python3.9/site-packages/fractal_tasks_core/tasks/_utils.py", line 79, in run_fractal_task
    metadata_update = task_function(**pars)
  File "/path/to/FRACTAL_TASK_DIR/.fractal/fractal-tasks-core1.1.0/venv/lib64/python3.9/site-packages/pydantic/v1/decorator.py", line 40, in wrapper_function
    return vd.call(*args, **kwargs)
  File "/path/to/FRACTAL_TASK_DIR/.fractal/fractal-tasks-core1.1.0/venv/lib64/python3.9/site-packages/pydantic/v1/decorator.py", line 134, in call
    return self.execute(m)
  File "/path/to/FRACTAL_TASK_DIR/.fractal/fractal-tasks-core1.1.0/venv/lib64/python3.9/site-packages/pydantic/v1/decorator.py", line 206, in execute
    return self.raw_function(**d, **var_kwargs)
  File "/path/to/FRACTAL_TASK_DIR/.fractal/fractal-tasks-core1.1.0/venv/lib/python3.9/site-packages/fractal_tasks_core/tasks/cellvoyager_to_ome_zarr_init_multiplex.py", line 198, in cellvoyager_to_ome_zarr_init_multiplex
    plate = dict_acquisitions["0"]["plate"]
KeyError: '0'
jluethi commented 3 months ago

Will be closed by https://github.com/fractal-analytics-platform/fractal-tasks-core/pull/812

We can't support arbitrary acquisition names, because the spec says that acquisitions need to be integers. But we now support arbitrary integers. Additionally, the task checks that those values are actually integers. Finally, the OME-Zarr image groups will still be called 0, 1, 2, 3 etc., independent of the acquisition integers. That will make downstream issues with tasks assuming that there will be an image 0 less likely. The well has the corresponding metadata to describe which image is which acquisition and the images are ordered by their acquisitions.