search

anegostudios / VintageStory-Issues

Vintage Story's public issue tracker for reporting bugs, crashes and the like
43 stars 16 forks source link

Multiplayer world save file is getting bigger and bigger #1684

Open petrnowak opened 2 years ago

petrnowak commented 2 years ago

Game Version: 1.16.3 Platform: Linux (Ubuntu) Modded: Yes (Medieval expansion, HUD clock, Fancy doors, ) SP/MP: Multiplayer

Description: Multiplayer world save file gets bigger and bigger. Our MP server world file does have over 33 GB now. We do full backup every day. We keep 7 backups to have possibility to return back in case of any serious problem. 7 backups are not much... We'd like to keep more. This means a lot of GBs !!! Backuping of world save file is getting more and more stressfull operation taking a lot of time. And we are 4 players only playing only a hour a day.

Don't you consider some incremental method of world backup? Including some settings - frequency of incremental backup and full backup. For example: incremental backup every day. Full backup once a week.

fipil commented 2 years ago

Some additional information: the /genbackup feature becomes more and more unusable for us. We play over a year on our VS server, now, as Petr wrote above, the data file size is over 33 GB. We urgently need some incremental backup or an additional tools for dealing with world map data regions/chunks. In the past years, we have operated a world on a Minecraft server for a couple of years. MC is storing a world data to region files, so we could backup only selected region files or we could delete some region files at all, to save space, because we knew that these regions can be regenerated when someone visits it again. VS does it differently, it stores it to an one big database, so it's a closed black box, it prevets us to do such things. As we'd like to play VS for long time and usually we keep playing on one world for a long time (many years), we need a backup solution, which is not so disc space expensive. I cannot imagine, that the backup file size will be 60GB, 80GB or more after a next year of playing. Please, this is a serious problem.

I'm a senior software developer, over 25 years in .NET. So if Tyron is too busy to do it, I offer myself as a contributor. Maybe I can help with that.

Aledark789 commented 2 years ago

Yea our world file on AuraFury is up to 125 gig, the compressed in tar.gz are roughly 40 gig each.

fipil commented 2 years ago

Okay, thanks @Aledark789 for the tip, I'm going to make a script to compress those backup files :-).

tels-git commented 2 years ago

Remember you can use pigz to compress files in parallel, and there is no need to put the default.vcdbs file in a tar file first, you can compress it straight away.

5HT2 commented 2 years ago

I'm currently working on an external tool to strip generated chunks from the DB in a radius / excluding protected zones, which would probably help with this

5HT2 commented 2 years ago

Seconding the points @fipil made, I'm running two vintagestory servers and they're using 8.1G / 20G respectively, and I'm only keeping a daily / weekly copy of each. This isn't really ideal, and the size used by each is rapidly expanding.