global: remove caching mechanism - Githubissues

reanahub / reana

REANA: Reusable research data analysis platform

https://docs.reana.io

MIT License

123 stars 54 forks source link

global: remove caching mechanism #788

Open mdonadoni opened 6 months ago

mdonadoni commented 6 months ago

REANA's current caching mechanism has been disabled by default for quite some time and it has some drawbacks:

big disk usage, as the whole workspace is copied every time a new job needs to be cached
slowness in job creation (see https://github.com/reanahub/reana-job-controller/pull/435)
caching does not always work as expected and some tricky cases are not handled correctly
this caching mechanism is only supported by serial workflows, and would not work with other engines as it does not handle multiple jobs running concurrently

For these reasons, we should consider removing this caching mechanism from REANA:

remove all related code
drop related columns in the database (migration script is needed)
update docs (reana/reana-client)
update API, e.g. do not accept CACHE option for serial workflows (breaking change!)
delete the cache directory from current deployments of REANA