Closed noib3 closed 2 months ago
Using LEB128 seems to bring a 0-10% reduction in message size compared to our own variable-length integer encoding algorithm.
Before:
bincode | Replica: 163.05 KB bincode | Total insertions: 3.24 MB bincode | Total deletions: 1.81 MB serde_json | Replica: 553.96 KB serde_json | Total insertions: 5.98 MB serde_json | Total deletions: 3.78 MB zstd'd bincode | Replica: 134.37 KB zstd'd bincode | Total insertions: 4.81 MB zstd'd bincode | Total deletions: 2.47 MB zstd'd serde_json | Replica: 156.90 KB zstd'd serde_json | Total insertions: 7.08 MB zstd'd serde_json | Total deletions: 4.42 MB
After:
bincode | Replica: 160.76 KB bincode | Total insertions: 3.14 MB bincode | Total deletions: 1.74 MB serde_json | Replica: 494.34 KB serde_json | Total insertions: 5.39 MB serde_json | Total deletions: 3.43 MB zstd'd bincode | Replica: 133.02 KB zstd'd bincode | Total insertions: 4.71 MB zstd'd bincode | Total deletions: 2.40 MB zstd'd serde_json | Replica: 146.91 KB zstd'd serde_json | Total insertions: 6.63 MB zstd'd serde_json | Total deletions: 4.09 MB
Using LEB128 seems to bring a 0-10% reduction in message size compared to our own variable-length integer encoding algorithm.
Before:
After: