Enable other packages reusing tasks from fractal-tasks-core compound tasks

[jluethi]

Especially relevant for reusing someone standard init tasks.

Example: Could a task in my_new_task_package consist of an init-task from fractal-tasks-core & a compute task from my_new_task_package? e.g. use the init for copy-ome-zarr-hcs-plate, but then do a min projection instead of a sum projection [we could of course put multiple projection algorithms in the core package task, more as an illustration of the use case]

Feedback Tommaso:

At the moment this is not possible. The quick and dirty workaround at the tasks-level is to have a task of your new package which imports and calls the fractal-tasks-core one. In the future we may modify the fractal-tasks-core dev tools and task-collection, but we'll need to think it through (how would a task developer declare the association to a task that belongs to a different package? TBD)

Excluded directions: Allow this to happen on the db side, e.g. reuse an existing part of a compound task from another task db object

[jluethi]

My initial thoughts on:

(how would a task developer declare the association to a task that belongs to a different package? TBD)

A task developer puts in something like this in their task_list:

    CompoundTask(
        name="Minimum Intensity Projection HCS Plate",
        input_types={"is_3D": True},
        executable_init="<fractal-tasks-core package dependeny>.copy_ome_zarr_hcs_plate.py",
        executable="tasks/minimum_intensity_projection.py",
        output_types={"is_3D": False},
    ),

But certainly something to be investigated

[tcompa]

The goal is clear, the implementation for the moment is TBD.

My first guess is that we would need to expand the options for what can be written in the manifest, in a way that refers to a different package. Then, we would need to modify the task-collection functions so that they identify this specific branch, and perform custom logic.

For context, current task collection functions transform "executable" strings into absolute-path commands by finding them in the current-package directory. The fractal-tasks-core files are not in the current-package directory, so that we need a way to find the correct package directory (through something like pip show), and reconstruct the correct path.

A similar logic would be needed in the task dev tools, because we would need to find the correct fractal-tasks-core Python modules so that we can recreate manifest entries and their schemas.

[tcompa]

A task developer puts in something like this in their task_list:

Agreed: the goal could be this kind of syntax (or maybe something slightly more complex - TBD).