search

beyond-all-reason / spring

A powerful free cross-platform RTS game engine
https://beyond-all-reason.github.io/spring/
Other
223 stars 102 forks source link

Compress replays with LZMA2 #505

Open 6AKU opened 2 years ago

6AKU commented 2 years ago

LZMA2 already being used for maps, so i suggest use it for replays too.

6AKU commented 2 years ago

Need tag "enhancement"

Beherith commented 2 years ago

SDFZ = spring demo file zipped so already done, you can decompress them if you want to look at the contents of the demo

6AKU commented 2 years ago

Already done and LZMA2 giving more compression rate.

lhog commented 2 years ago

Please unzip the replay and then re-archive it with 7z (make two experiments with normal and maximum compression ratios), then compare with sdfz size. I'm curious how the compression will perform on a single file.

6AKU commented 2 years ago

Results: Снимок

Profiles: Снимок2

lhog commented 2 years ago

I checked and the only hurdle seems to be the need to deal with insane LZMA API, the rest is pretty extensible. That's ~ a day worth of efforts, so unfortunately can't be put together quickly.

sprunk commented 1 year ago

What exactly was done to obtain the results above? Individual replays aren't 2-3 GB in size. If you unpack a lot of individual replays and then compress the whole folder into a single archive (maybe even if you compress a whole folder of already-individually-compressed replays too?) you'll naturally get better results because that enables the compression of things across replays (for example the startscript section is fairly unique data in a given replay but it tends to look similar in multiple replays).