search

gamercade-io / gamercade_console

A Neo-Retro Fantasy Console. Make WASM-powered, networked multiplayer games.
https://gamercade.io
Apache License 2.0
165 stars 10 forks source link

Investigate `bitcode` vs `bincode` for filesystem #99

Open RobDavenport opened 1 year ago

RobDavenport commented 1 year ago

Currently, we use bincode.

bitcode was recently released which has better compression (size) in exchange for slightly slower speeds. Could be benefecial to switch over if the gains are big enough.

RobDavenport commented 5 months ago

Recent benchmarks:

https://david.kolo.ski/rust_serialization_benchmark/

https://www.reddit.com/r/rust/comments/13hwwhj/bitcode_04_release_binary_serialization_format/

This seems like it would be worth it now.

RobDavenport commented 5 months ago

Interestingly I ran a test with this on the 3d example and somehow came up with a bigger file:

  1. bincode + zstd: 143kb
  2. bitcode raw: 443kb
  3. bitcode + zstd: 168kb
RobDavenport commented 2 months ago

Closing this for now, seems unnecessary to continue exploring this at this point

caibear commented 2 months ago

bitcode author here, your test on Feb 4 was with an old version of bitcode (presumably 0.4 or 0.5). These old versions didn't compress very well because they serialized bits while compression like zstd operates on bytes. Luckily I have released a new version bitcode = "0.6" which compresses much better.

Using version 3de7156db381dd2fa1943d59551360819e3e27f7 because the latest version failed to deserialize, I ran some tests on example_3d.gcrom. I initially measured a small regression from bincode 160kb -> 164kb. However, after further investigation I managed to get an improvement to 115kb with a small code change.

I replaced https://github.com/gamercade-io/gamercade_console/blob/31bec1529c8b828c762a7c1044b7933deab10a19/gamercade_core/src/graphics/color.rs#L20-L54

with #[derive(Serialize, Deserialize)] which serializes the color as (u8, u8, u8, u8) instead of u32. This allows bitcode to leverage its improved compression (but has no effect on bincode). Assuming you want to keep the human readable representation, I recommend using (self.r, self.g, self.b, self.a).serialize(serializer) to achieve the same effect.

RobDavenport commented 2 months ago

Thanks for looking into this. Going to re-open the issue as it will be worth it to look into it again in the future!