search

conformal / cyphertite

A high-security scalable solution for online backups.
https://www.cyphertite.com/
46 stars 8 forks source link

cull does not respect ctfile_cachedir_max_size and fails when disk fills up #98

Open zman0900 opened 9 years ago

zman0900 commented 9 years ago

Host: Arch Linux Version: cyphertite 2.0.2 (built from https://aur.archlinux.org/packages/cyphertite-external-libs)

I have set ctfile_cachedir_max_size = 5G in my config file, but when I run cyphertitectl cull, it fills up the partition that has over 20 GB of free space. This seems to cause the cull to just fail and not delete anything, because cyphertite -mt shows I have both full and incremental ct files going all the way back to March even though ctfile_cull_keep_days was set to 90.

When I sum up all the file sizes output from cyphertite -mt, they only add to about 11 GB, so it seems strange that downloading them all to the cache would take more than 20 GB.

Expected outcome

Running cyphertitectl cull should stop downloading stuff to the cache directory when it reaches its max configured size, or the disk becomes full.

Additional info

I have changed ctfile_cull_keep_days to 30 and mounted an empty 100 GB partition to /root/.cyphertite/ct_cachedir and am currently running another cull to see if it will at least clean out the old data when given way too much space.

Here's my censored config file:

host                                    = auth.cyphertite.com
hostport                                = 48879
username                                = zman0900
password                                = <redacted>
crypto_passphrase                       = <redacted>
cache_db                                = /root/.cyphertite/ct_db
session_compression                     = lzo
crypto_secrets                          = /root/.cyphertite/ct_crypto
ca_cert                                 = /root/.cyphertite/ct_certs/ct_ca.crt
cert                                    = /root/.cyphertite/ct_certs/ct_zman0900.crt
key                                     = /root/.cyphertite/ct_certs/private/ct_zman0900.key
ctfile_mode                             = remote
ctfile_cachedir                         = /root/.cyphertite/ct_cachedir
ctfile_cachedir_max_size                = 5G
ctfile_remote_auto_incremental          = 1
ctfile_max_incrementals                 = 6
ctfile_cull_keep_days                   = 30
upload_crypto_secrets                   = 1

My full backups are ran mondays at 6pm with:

cyphertite -c0Rf nas.main -I /root/.cyphertite/nas.main-include.txt -E /root/.cyphertite/nas.main-exclude.txt /

Incrementals are ran Tuesday-Sunday at 6pm with:

cyphertite -cRf nas.main -I /root/.cyphertite/nas.main-include.txt -E /root/.cyphertite/nas.main-exclude.txt /

Cull is ran Tuesdays at 2pm with:

cyphertitectl cull

Additional backups run on a similar schedule offset by 12 hours on a different machine.

zman0900 commented 9 years ago

To update, the cull that I started with the larger 100 GB partition for the cache directory did eventually complete and was successful. But it took somewhere between 12 and 18 hours and the total cache space used was over 30 GB when I checked at around 12 hours. That is with only about 280 GB of backup space used.