search

SF-Sudio / ServerBackup

GNU General Public License v3.0
38 stars 12 forks source link

Allow different formats / compression types #4

Open 0xicl33n opened 2 years ago

0xicl33n commented 2 years ago

Personally i would love to have the option to use 7z files and zstd for compression, but other people may want rar, tar, tar.gz, etc

It would be fantastic to have the option! Would seriously help on server hard drive space with big worlds

SF-Sudio commented 2 years ago

Thank you for your intersting suggestion! I will definetly inform myself about how to implement other compression formats and add this feature in one of the next updates.

NyaomiDEV commented 2 years ago

Some tests with my 261.8 MiB world (if you're on Windows, MiB is basically MB).

All the tests were done via KDE's Ark program on a Ryzen 3800XT with some DDR4 RAM clocked at 3200MHz, just to have some more data about baseline PC performance.

Zip (this plugin): 221.2MiB Tar + Gzip (average compression rate): 220.1MiB Tar + Bzip2 (max compression rate): 219.5MiB (took a fair bit of time, wasn't worth it) Tar + XZ (average compression rate): 209.0MiB (took more time than the above though) Tar + Zstandard (min compression rate, super fast): 217.4MiB Tar + Zstandard (average compression rate): 210.4MiB Tar + Zstandard (max compression rate, took some time): 207.4MiB 7zip + LZMA2 (min compression rate): 220.8MiB 7zip + LZMA2 (average compression rate): 208.8MiB 7zip + LZMA2 (max compression rate): 207.8MiB (and it took way more time than the above, for minimum gain)

The best compromise I had with 7z+LZMA2 average, since it didn't take so long and I was below 210MB per backup. It is a 12.4MiB gain over standard zip, which can definitely pile up to a massive space gain if you plan to do a lot of backups per day (once every half an hour for example). The extreme best was Tar + Zstandard at max compression rate, though it took a fair bit (but it is negligible if we count the async nature of the backup, and it was less time than Tar + XZ, actually comparable to Tar + Bzip2).

So yeah 7z compression does make sense for Minecraft worlds.

SF-Sudio commented 2 years ago

That's some great data to work with, thank you! :)

0xicl33n commented 2 years ago

How is the compression with larger worlds? I run a few big servers for a couple content creators and one of our world folders is 32gb.

NyaomiDEV commented 2 years ago

How is the compression with larger worlds? I run a few big servers for a couple content creators and one of our world folders is 32gb.

I don't have a 32gb world folder to test with; but you could run some benchmarks yourself. Ruling out the blatantly unsuccessful compression methods, you can try Tar+Zst and 7z + LZMA2