Megathread following discussion in Enigmatica Modpacks

MommyHeather commented 12 months ago

Issue made after a discussion in this support thread

[x] 1. Have a way to restore backups from within the client - ideally a menu option on the main menu / world list.
[x] 2. Stick a README-BEFORE-RESTORING.txt in the backups root dir - inform users of how to use the tool, and how to restore manually.
[x] 3. Print what mode is currently in use when making a backup.
[x] 4. Polish up default configs - ideally, default for the mod can be reworked to what we think would be best as a default for modpacks
[x] 5. Review existing issues - particular attention to #3

MommyHeather commented 12 months ago

resolved in 33433c1dea686a1afc825ae10ce0b5e9837d2c6b
resolved in 00fc21bf21cf5988837c022dd5b091ecfd5b9e6b
input wanted - @NielsPilgaard
I believe is also resolved in 00fc21bf21cf5988837c022dd5b091ecfd5b9e6b

MommyHeather commented 12 months ago

will likely NOT be a gui option - doing so will be a lot of work given I plan to keep support for all versions that I currently or will support. I think it will be WAY easier to make a system call to run the cli tool with the jvm currently running the client.

MommyHeather commented 12 months ago

A simple bat or sh script could solve 1. - see #10 for more info

NielsPilgaard commented 12 months ago

input wanted - @NielsPilgaard

Would you mind pasting the current default config here? :)

MommyHeather commented 12 months ago

Would you mind pasting the current default config here? :)

config

``` #Enable or disable automatic backups. #Options : true, false #Default : true, config.advancedbackups.enabled=true #Whether to save before making a backup. #Options : true, false #Default : false config.advancedbackups.save=false #Whether to require player activity between backups. #Options : true, false #Default : false config.advancedbackups.activity=false #The type of backups to use. #Options : zip, differential, incremental #Default : differential config.advancedbackups.type=differential #The absolute or relative path to the backup location. #Options : any file path. Default : ./backups config.advancedbackups.path=./backups #The maximum size to keep, in GB. Keep relatively high for zips, tighter space requirements should instead use differential or incremental backups. #Range : 5 - 9999 #Default : 50 config.advancedbackups.size=50 #Minimum time between backups, in hours. This can prevent a shutdown backup from triggering immediately after a scheduled backup or similar situations. #Range : 0.5 - 500 #Default : 0.5 config.advancedbackups.frequency.min=0.5 #Triggers a backup if none has already happened within this time. Can be combined with an uptime-based schedule. #Range : 0.5 - 500 #Default : 24 config.advancedbackups.frequency.max=24 #Whether the schedule below uses uptime (true) or real-world time (false). #Default : true config.advancedbackups.frequency.uptime=true #When using server uptime: #A looping comma-separated backup schedule, based off of server uptime, hours:minutes. Examples: #4:00 - Makes a backup every four hours. #4:00,7:00 - Makes a backup after four hours, then three, then four, and so on. #1:00 - Makes a backup every hour. #4:00,8:00,12:00,16:00,17:00,18:00,19:00,20:00,21:00,24:00 - Makes a backup following a strict schedule. #When using real-world time: #A strict schedule, using hours:minutes to follow real-world time. Examples: #4:00 - Makes a backup at 4am each day. #4:00,8:00,12:00,16:00,17:00,18:00,19:00,20:00,21:00,24:00 - Makes a backup at specific times of day. #Default : 12:00 config.advancedbackups.frequency.schedule=12:00 #Whether to force a backup on server shutdown. Respects min frequency. #Options : true, false #Default : false config.advancedbackups.frequency.shutdown=false #Whether to force a backup on server startup. Respects min frequency. #Options : true, false #Default : false config.advancedbackups.frequency.startup=false #Delay to use after startup, in seconds. Is always at least 5 seconds. #Range : 5-9999999999 config.advancedbackups.frequency.delay=5 #Whether to disable console and chat logging. Does not affect debug.log, does not affect error messages. #Options : true, false #Default : false config.advancedbackups.logging.silent=false #-------------------------------------------------------------------------------------------------------------------- ##The following options only affect zip files, whether that's for zip backups, export commands or some other option. #-------------------------------------------------------------------------------------------------------------------- #The compression level to use for zip files. Higher numbers space usage, but decrease performance. #Range : 1-9 #Default : 4 config.advancedbackups.zips.compression=4 #-------------------------------------------------------------------------------------------------------------------- ##The following options only affect differential and incremental backups. #-------------------------------------------------------------------------------------------------------------------- #The maximum 'chain' length to keep. #Range : 5-500 #Default : 50 config.advancedbackups.chains.length=50 #Whether to compress 'chains'. This compresses the base backup and all sequential backups. Reduces space usage, but decreases performance. #Options : true, false #Default : true config.advancedbackups.chains.compress=true #Whether to enable "smart" reset for chains - if every file is being backed up, mark the backup as complete and reset chain length regardless of intended backup type. #Options : true, false #Default : true config.advancedbackups.chains.smart=true #What % of a full backup is allowed to be contained in a partial before forcing it into a full backup. Useful for reducing partial backup size. config.advancedbackups.chains.maxpercent=75 #Whether to delete incremental backup chains if max size is exceeded. If not, incremental backups do not respect the max size config and never delete. #Options : true, false #Default : false config.advancedbackups.purge.incrementals=false ```

There are changes I want to make in terms of descriptions, defaults and available options, but I'm not really 100% sure on anything atm.

The ranges listed aren't actually enforced currently but will be soon - they can of course be adjusted

NielsPilgaard commented 12 months ago

Here are the values I propose as defaults, I think these would suit most modpacks:

#Enable or disable automatic backups.
#Options : true, false   #Default : true,
config.advancedbackups.enabled=true

#Whether to save before making a backup.
#Options : true, false    #Default : false
config.advancedbackups.save=true

#Whether to require player activity between backups.
#Options : true, false    #Default : false
config.advancedbackups.activity=true

#The type of backups to use.
#Options : zip, differential, incremental    #Default : differential
config.advancedbackups.type=differential

#The absolute or relative path to the backup location.
#Options : any file path. Default : ./backups
config.advancedbackups.path=./backups

#The maximum size to keep, in GB. Keep relatively high for zips, tighter space requirements should instead use differential or incremental backups.
#Range : 5 - 9999   #Default : 50
config.advancedbackups.size=50

#Minimum time between backups, in hours. This can prevent a shutdown backup from triggering immediately after a scheduled backup or similar situations.
#Range : 0.5 - 500    #Default : 0.5
config.advancedbackups.frequency.min=0.5

#Triggers a backup if none has already happened within this time. Can be combined with an uptime-based schedule.
#Range : 0.5 - 500    #Default : 24
config.advancedbackups.frequency.max=24

#Whether the schedule below uses uptime (true) or real-world time (false).
#Default : true
config.advancedbackups.frequency.uptime=true

#When using server uptime:
    #A looping comma-separated backup schedule, based off of server uptime, hours:minutes. Examples:
    #4:00 - Makes a backup every four hours.
    #4:00,7:00 - Makes a backup after four hours, then three, then four, and so on.
    #1:00 - Makes a backup every hour.
    #4:00,8:00,12:00,16:00,17:00,18:00,19:00,20:00,21:00,24:00 - Makes a backup following a strict schedule.

#When using real-world time:
    #A strict schedule, using hours:minutes to follow real-world time. Examples:
    #4:00 - Makes a backup at 4am each day.
    #4:00,8:00,12:00,16:00,17:00,18:00,19:00,20:00,21:00,24:00 - Makes a backup at specific times of day.

#Default : 12:00
config.advancedbackups.frequency.schedule=1:00

#Whether to force a backup on server shutdown. Respects min frequency.
#Options : true, false    #Default : false
config.advancedbackups.frequency.shutdown=false

#Whether to force a backup on server startup. Respects min frequency.
#Options : true, false    #Default : false
config.advancedbackups.frequency.startup=false

#Delay to use after startup, in seconds. Is always at least 5 seconds.
#Range : 5-9999999999
config.advancedbackups.frequency.delay=30

#Whether to disable console and chat logging. Does not affect debug.log, does not affect error messages.
#Options : true, false    #Default : false
config.advancedbackups.logging.silent=false

#--------------------------------------------------------------------------------------------------------------------
##The following options only affect zip files, whether that's for zip backups, export commands or some other option.
#--------------------------------------------------------------------------------------------------------------------

#The compression level to use for zip files. Higher numbers space usage, but decrease performance.
#Range : 1-9    #Default : 4
config.advancedbackups.zips.compression=4

#--------------------------------------------------------------------------------------------------------------------
##The following options only affect differential and incremental backups.
#--------------------------------------------------------------------------------------------------------------------

#The maximum 'chain' length to keep.
#Range : 5-500    #Default : 50
config.advancedbackups.chains.length=50

#Whether to compress 'chains'. This compresses the base backup and all sequential backups. Reduces space usage, but decreases performance.
#Options : true, false    #Default : true
config.advancedbackups.chains.compress=true

#Whether to enable "smart" reset for chains - if every file is being backed up, mark the backup as complete and reset chain length regardless of intended backup type.
#Options : true, false    #Default : true
config.advancedbackups.chains.smart=true

#What % of a full backup is allowed to be contained in a partial before forcing it into a full backup. Useful for reducing partial backup size.
config.advancedbackups.chains.maxpercent=50

#Whether to delete incremental backup chains if max size is exceeded. If not, incremental backups do not respect the max size config and never delete.
#Options : true, false    #Default : false
config.advancedbackups.purge.incrementals=true

A few questions:

When config.advancedbackups.chains.length is exceeded, will the oldest non-full incremental backup be deleted, or what happens?
I'm unsure about config.advancedbackups.purge.incrementals - If this is true, will the oldest entire chain be deleted if max size is exceeded?

MommyHeather commented 12 months ago

When config.advancedbackups.chains.length is exceeded, will the oldest non-full incremental backup be deleted, or what happens?

The chain resets - meaning the next backup is a full backup, and new incrementals or differentials will be based off of that one. This is especially important for incrementals, as one slight blip in the chain can ruin the entire chain.

This value does not delete anything - lower values offer more frequent full backups but increase space usage.

I'm unsure about config.advancedbackups.purge.incrementals - If this is true, will the oldest entire chain be deleted if max size is exceeded?

Yes, this is the case - as with incrementals, if even a single backup in a chain is lost, every backup between that one and the end of that chain is useless.

With differentials however, partial backups can be lost without affecting other partial backups - so they're removed before the fill backup they depend on in a case of size being exceeded.

I hope i did a good job explaining there hehe

NielsPilgaard commented 12 months ago

I hope i did a good job explaining there hehe

You did, thanks 👍 I stand by my recommendation then. I'm however unsure if we can decrease the default max size - 50GB is a bit much, but with normal zips it's usually a good minimum. Time will have to tell I guess.