inveniosoftware / invenio-files-rest

REST API for uploading/downloading files for Invenio.
https://invenio-files-rest.readthedocs.io
MIT License
9 stars 67 forks source link

The `clear_orphaned_files` task cannot be scheduled like the documentation says #290

Open max-moser opened 1 year ago

max-moser commented 1 year ago

Package version (if known): 1.5.0

Describe the bug

The clear_orphaned_files task specifies that i can be scheduled by adding a configuration entry to the CELERY_BEAT_SCHEDULE like so:

from invenio_app_rdm.config import CELERY_BEAT_SCHEDULE

CELERY_BEAT_SCHEDULE["clear-orphan-files"] = {
  "task": "invenio_files_rest.tasks.clear_orphaned_files",
  "schedule": 60 * 60 * 24,
  "kwargs": {
    "force_delete_check": lambda file: False,
    "limit": 500,
  }
}

However, this doesn't seem to work because the force_delete_check argument (lambda file: False) can't be pickled. We need to find an alternative way of how to pass the check to the task in a pickle-friendly way. I'm thinking of a classic object_or_import_string here, maybe there's better alternatives though.