search

fractal-analytics-platform / fractal-server

Fractal backend
https://fractal-analytics-platform.github.io/fractal-server/
BSD 3-Clause "New" or "Revised" License
11 stars 3 forks source link

Make a copy of wheel file in task-collection #1881

Open tcompa opened 4 weeks ago

tcompa commented 4 weeks ago

When a user provides the path to a wheel file, it could be e.g. /random/dir/fractal_tasks_mock-0.0.1-py3-none-any.whl. Before the actual pip install operation, we should

  1. Copy the wheel file to the actual task-collection folder (e.g. /somewhere/{user_id}/fractal-tasks-mock/0.0.1/).
  2. Update the corresponding TaskGroupV2.wheel_path.
  3. pip install from there.

This way, the pip-freeze will include data like this

fractal-tasks-mock @ file:///somewhere/{user_id}/fractal-tasks-mock/0.0.1//fractal_tasks_mock-0.0.1-py3-none-any.whl
pydantic==1.10.18
typing_extensions==4.12.2

which will be reproducible even after deactivation.

TLDR: we cannot rely on the user-provided wheel to still exist, when we'll have to re-activate the venv.

tcompa commented 1 day ago

See https://github.com/fractal-analytics-platform/fractal-server/issues/1997 for folder structure

tcompa commented 1 day ago

Steps:

  1. Within background local task collection, if wheel_path is given, copy that file (via shutil.cp) to taskgroup.path
  2. Then replace taskgroup.wheel_file with the new path.
  3. Proceed as usual.

Note: step 1 must be executed after the mkdir statements.