search

fractal-analytics-platform / fractal-tasks-core

Main tasks for the Fractal analytics platform
https://fractal-analytics-platform.github.io/fractal-tasks-core/
BSD 3-Clause "New" or "Revised" License
14 stars 6 forks source link

Role of `overwrite` in `cellvoyager_to_ome_zarr_compute.py` #720

Closed tcompa closed 7 months ago

tcompa commented 7 months ago

When I set overwrite=True in cellvoyager_to_ome_zarr_init and overwrite=False in cellvoyager_to_ome_zarr_compute, the compound task runs through with no errors.

I need to debug this more carefully, but I think it is due to the init task recreating empty zarr groups for the wells - so that the compute task doesn't need the overwrite flag.

At some point we can decide that the compute task only runs after a successful init task, and then we don't need to expose the same parameter again -> TBD

This is a pattern that we'll hit again, with compound task (something like "do the init/compute tasks need to share some information?", or even "the compute task can/cannot rely on assumptions about what happened within the init task").

jluethi commented 7 months ago

Yes, I came across the same here: https://github.com/fractal-analytics-platform/fractal-tasks-core/issues/699

Closing that one as duplicate.

For the use in Fractal, we should just drop the overwrite parameter in the second task. For use outside of Fractal, there still could be a reason to have it, but I don't think it's a very strong case.

jluethi commented 7 months ago

Let's drop them now & add it to the task docstring