search

itzg / docker-mc-backup

Provides a side-car container to backup itzg/minecraft-server world data
https://hub.docker.com/r/itzg/mc-backup
MIT License
298 stars 51 forks source link

Attempt to add rsync method #165

Closed toddejohnson closed 7 months ago

toddejohnson commented 7 months ago

This is my attempt to add #90 rsync support as a backup method. This also hard links the duplicate files to reduce needed storage based upon the last backup. This is not using the LINK_LATEST sadly as the hard link is a important feature if we don't have compression.

I've yet to test! Why let testing hold back a starting point. I hope to test soon as I get this re-built and pushed somewhere my cluster can pull from.

toddejohnson commented 7 months ago

I think the --link-dest can do that or incremental but that would require a full

itzg commented 7 months ago

I see what you mean. Cool. Let me know when the testing is finished and I'll be good to merge this.

toddejohnson commented 7 months ago

I'm trying to get the helm chart to use it. Guess I need a PR there too as rsync isn't in the schema. I bypassed helm and it is running ~5GB set to my NFS target. Yay!

toddejohnson commented 7 months ago

Here are my test results, Full copy(looks like ~500M after exclude jar:

2023-12-22T07:16:50+0000 INFO waiting initial delay of 2m...
2023-12-22T07:18:50+0000 INFO waiting for rcon readiness...
2023-12-22T07:18:50+0000 INFO Command executed successfully rcon-cli save-on
2023-12-22T07:18:50+0000 INFO Command executed successfully rcon-cli save-off
2023-12-22T07:18:50+0000 INFO Command executed successfully rcon-cli save-all flush
2023-12-22T07:18:50+0000 INFO Command executed successfully sync
2023-12-22T07:18:50+0000 INFO No previous backups. Running full
2023-12-22T07:18:50+0000 INFO Backing up content in /data to /backups/world-20231222-071850
2023-12-22T07:20:40+0000 INFO Command executed successfully rcon-cli save-on
2023-12-22T07:20:40+0000 INFO sleeping 24h...

Incremental using hardlink(~13M):

2023-12-22T07:25:12+0000 INFO waiting initial delay of 2m...
2023-12-22T07:27:12+0000 INFO waiting for rcon readiness...
2023-12-22T07:27:12+0000 INFO Command executed successfully rcon-cli save-on
2023-12-22T07:27:12+0000 INFO Command executed successfully rcon-cli save-off
2023-12-22T07:27:13+0000 INFO Command executed successfully rcon-cli save-all flush
2023-12-22T07:27:13+0000 INFO Command executed successfully sync
2023-12-22T07:27:13+0000 INFO Latest found so using it for link
2023-12-22T07:27:13+0000 INFO Backing up content in /data to /backups/world-20231222-072713
2023-12-22T07:27:23+0000 INFO Command executed successfully rcon-cli save-on
2023-12-22T07:27:23+0000 INFO sleeping 24h...

Removed latest to force guessing:

rminated
time="2023-12-22T07:34:53Z" level=info msg="Skipping uid/gid change since current user is not root"
2023-12-22T07:34:54+0000 INFO waiting initial delay of 2m...
2023-12-22T07:36:54+0000 INFO waiting for rcon readiness...
2023-12-22T07:36:54+0000 INFO Command executed successfully rcon-cli save-on
2023-12-22T07:36:54+0000 INFO Command executed successfully rcon-cli save-off
2023-12-22T07:36:54+0000 INFO Command executed successfully rcon-cli save-all flush
2023-12-22T07:36:54+0000 INFO Command executed successfully sync
2023-12-22T07:36:54+0000 INFO Searching for latest backup to link with
2023-12-22T07:36:54+0000 INFO Backing up content in /data to /backups/world-20231222-073654
2023-12-22T07:37:04+0000 INFO Command executed successfully rcon-cli save-on
2023-12-22T07:37:04+0000 INFO sleeping 24h...

Should be good for review/testing. I'm going to look at the helm charts tomorrow.