search

MommyHeather / AdvancedBackups

BSD 3-Clause "New" or "Revised" License
26 stars 4 forks source link

Feature request : Add option to specify differential chains ending automatically if the differential backup exceeds a certain size #6

Closed MisterMyst closed 1 year ago

MisterMyst commented 1 year ago

Let's say a player goes off and explores and generates a whole bunch of new chunks, inflating the size of the differential backup. Every partial backup after this will be larger than it needs to be, since those chunks are not likely to change much again. This will likely be extra common during the early game, and the longer the game goes on, the longer the chains can be without being inefficient on average.

I think a great solution to this is to have an option to force the differential to create a new chain instead of continuing the current chain, if the last partial backup exceeded a certain percentage of the full backup size. I'd suggest somewhere between 20-40% perhaps. Alternatively can make it a specified size, but I think a percentage makes more sense in the interest of reducing waste with partial backups

Alternatively, it could also use a percentage of the mean partial size, rather than a percentage of the full size. This may make for a better heuristic overall, hard to say for sure. Say the newest partial is 150% of the average partial size INCLUDING the current partial. This would mean a sudden influx of changes which may not be representative of what needs to be backed up on average, and therefore the partials are not very efficient anymore.

There should probably also be a minimum chain length as well to enforce, especially if the average of partials is tried instead of percentage of full.

Another potential option is to have the chain restart after the sum of partials exceeds some percentage of the full. Like if the sum of partials is greater than 100% of the full backup, maybe it's time to make a new chain.

With any of these options, the dynamic chain length would ensure we don't become overly redundant with the partial backups and should save quite a bit of space in many cases if tuned properly

MommyHeather commented 1 year ago

My thoughts on this :

Another potential option is to have the chain restart after the sum of partials exceeds some percentage of the full. Like if the sum of partials is greater than 100% of the full backup, maybe it's time to make a new chain.

A config value for this, alongside a command to manually reset the chain length, would be the best option imo?

MommyHeather commented 1 year ago

I've come to feel like comparing the current backup to that same backup if it would be partial is the better idea, so that's what I've done.