lancachenet / monolithic

A monolithic lancache service capable of caching all CDNs in a single instance
https://hub.docker.com/r/lancachenet/monolithic
Other
737 stars 73 forks source link

Managing Downloaded Files #15

Closed DarrenRainey closed 3 years ago

DarrenRainey commented 5 years ago

Describe the issue you are having

Can you add a way to easily manage/view what's stored on the cache to make it easier to find and fix corrupted files as well as delete old/unused files.

How are you running the container(s)?

HOST_IP=`hostname -I | cut -d' ' -f1`
docker run --restart unless-stopped --name steamcache-dns --detach -p 53:53/udp -e USE_GENERIC_CACHE=true -e LANCACHE_IP=$HOST_IP steamcache/steamcache-dns:latest
docker run --restart unless-stopped --name lancache --detach -v /mnt/HDD1/cache/data:/data/cache -v /mnt/HDD1/cache/logs:/data/logs -p 80:80 -e CACHE_MEM_SIZE=12000m -e CACHE_DISK_SIZE=930g steamcache/monolithic:latest
docker run --restart unless-stopped --name sniproxy --detach -p 443:443 steamcache/sniproxy:latest
echo Please configure your dhcp server to serve dns as $HOST_IP

DNS Configuration

Google DNS
VibroAxe commented 5 years ago

So the short answer is no, because unlike the steamcache cache monolithic uses nginx' internal cache mechanic (because it's more efficient). The downside of this is that it stores the cached files using a hash instead of the file name.

We don't really see corrupted files in the same way that steamcache used to due to the way the nginx cache works, however we are looking at a way to do a forward lookup to produce the hashes for a known file/chunk.

Leave it with us

DarrenRainey commented 5 years ago

Starting to work on this as well my self but so far the simplest solution seems to be this on the cache/hard drive side:

for delete in  $(grep -r "steam/depot/441" * | awk '{print $3}')
do
rm -rf $delete
done

although I need to do more testing and see how nginx handes the file changes as well as speed optimisations.

It0w commented 5 years ago

Hi i use https://github.com/perusio/nginx-cache-purge and have found it here: https://github.com/bntjah/lancache/issues/112