search

fractal-analytics-platform / fractal-tasks-core

Main tasks for the Fractal analytics platform
https://fractal-analytics-platform.github.io/fractal-tasks-core/
BSD 3-Clause "New" or "Revised" License
14 stars 6 forks source link

Could we only include the `tasks` subpackage when `fractal-tasks` extra is installed? #531

Closed tcompa closed 2 months ago

tcompa commented 11 months ago

I'm not sure that what I propose below is feasible, but it would be useful.

EDIT: unfortunately I'm afraid this is the same as https://github.com/fractal-analytics-platform/fractal/issues/545, and then not easily feasible. Let's think about other options.

This is a bit of a packaging glitch: installing fractal-tasks-core without any extra does not install any of the required dependencies for the fractal_tasks_core.tasks subpackage, but the subpackage itself is still available. This may lead to

>>> from fractal_tasks_core.tasks import cellpose_segmentation
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/tmp/venv/lib/python3.10/site-packages/fractal_tasks_core/tasks/cellpose_segmentation.py", line 26, in <module>
    import cellpose
ModuleNotFoundError: No module named 'cellpose'

On top of being a bit counterintuitive in itself, this leads to errors like: https://github.com/fractal-analytics-platform/fractal-tasks-core/pull/524#issuecomment-1725915617. Fractal-server only checks that task files exist, but it does not try to load them and check their imports. Thus, at the moment, there is no way for fractal-server to know that something went wrong.

If the tasks subpackage were part of the extra, then the task files would not exist and fractal-server would raise an error.

jluethi commented 11 months ago

Interesting thoughts. I don't think this is something we need to prioritize for now

tcompa commented 2 months ago

Fractal-server only checks that task files exist, but it does not try to load them and check their imports.

It turns out that this is actually useful when the goal is to test task collection (rather than actually running tasks) in a new deployment or with a new configuration. In the current situation, the no-extra task collection will still succeed (and run much faster than the torch-inlcuding one).

Closing for now.