Open giorgiosironi opened 7 years ago
However, it's not possible to add the same atime
filter to the empty directories, as the find
operations will have set it to a recent time.
I guess we could download the file to the tmp directory, and then create the cache dir and move the file in without having the requests call in-between. I think there would still technically be a race condition, though.
Would mitigate, but the same problem reappears between the creation and the movement. If I find a solution I'll post it here.
My solution is to create two servers behind a load balancer, so that when performing cache cleaning the server can be detached first to avoid interference from real traffic: https://github.com/elifesciences/iiif-formula/blob/master/salt/iiif/config/usr-local-bin-loris-cache-clean It's not a portable solution in general, but someone may find it useful.
https://github.com/loris-imageserver/loris/blob/development/bin/loris-http_cache_clean.sh#L59 removes empty directories. However, if the cache clean runs while traffic is being served, there is a race condition with the creation of temporary files:
https://github.com/loris-imageserver/loris/blob/development/loris/resolver.py#L317 creates the cache dir, while https://github.com/loris-imageserver/loris/blob/development/loris/resolver.py#L331 creates the file in it. When the directory is removed between these two operations, the request fails with this exception.