Open riedel opened 2 years ago
yes, that is something that was considered a few times. I will have a look again.
I also got several request for an easy way to "delete" a workspace - not just releasing it - especially on systems where quotas are applied.
I just created a ws_delete
script, which performs file deletion and ws_release
in one command. The actual method used to delete the files and directories in the workspace might get tailored to the used filesystem, later, e.g., to apply a deletion rate limit. For the first go I used the rsync
method for fastest deletion.
See #98
ws_release --delete-data it has a warning and a 5 second delay to interrupt. if it gets interrupted, the workspace is released and recoverable. If it is not interrupted, data is deleted and the workspace is gone and not recoverable, DB entry is wiped as well.
For reference: Commit that first implemented this functionality: ac912be6243526539e1e0a5b883ced21e96386d2
I guess it's open for testing and feedback ;)
Background: ws_release does not recover quota immediately, which is a design decision. However, manually triggering deletion particularly when dealing with a lot of files requires a lot of care (e.g. using
lfs find
to not trigger a lot of metadata reads).It would be great if ws_release could provide an optional flag for immediate deletion.
Ideally this could be done not as part of the running main job (as this waste a lot of resources). However, it would be also OK if it is blocking because it at least caring about things (as another problem I concretely have is that my enroot containers creates files that are undeletable and need a change of permissions first)
Edit: As clarification: My use case might be a bit atypical: I need the workspace as ephemeral storage during a longer job. The job result is much smaller, why I can release the workspace at the end of the job automatically.