radiocosmology / alpenhorn

Alpenhorn is a service for managing an archive of scientific data.
MIT License
2 stars 1 forks source link

fix(LustreHSM): Move ready_pull I/O out of main thread #187

Closed ketiltrout closed 2 months ago

ketiltrout commented 2 months ago

This is the second part of the fix for #183 which fixes how the ready_pull function works. All I/O for ready_pull is now in a task and out of the main thread.

In the previous PR I added some slap-dash "race condition insurance" which I've removed here and replaced with a more robust system to restore-and-wait for a file which provides a way to track files already being waited on for restore. (Which was otherwise impossible to do because yielded tasks are essentially hidden from the system while they're deferred.)

Now checks and ready_pulls won't be re-queued if they're already waiting for a file to be restored.

Closes #183