dimo414 / bkt

a subprocess caching utility, available as a command line binary and a Rust library.
https://www.bkt.rs
MIT License
251 stars 13 forks source link

Possible Race When Starting Many Parallel Bkt Instances #38

Closed eatnumber1 closed 1 year ago

eatnumber1 commented 1 year ago

I use bkt in my .zshenv, which, among other things, runs on every parallel arm of a GNU parallel invocation. So I'm starting bkt in parallel many times.

Sometimes, I see:

bkt: Cache lookup failed: Failed to remove expired data: No such file or directory (os error 2)

Could this be a race between bkt invocations deleting expired data? Seems like something that could be made idempotent (... by ignoring ENOENT)?

dimo414 commented 1 year ago

Here's the offending block for reference. It's presumably racing with a background cleanup operation. The latter uses a simplistic locking mechanism to avoid races but the former does not. The cleanup loop already ignores deletion errors.

So it's probably fine to just disregard these errors, as you say. Alternatively it could check the same lock and skip if it's held. Any interest in contributing a fix? 😄