nomad / cola

🥤 A text CRDT for real-time collaborative editing
https://crates.io/crates/cola
MIT License
493 stars 12 forks source link

Encode integers using LEB128 #10

Closed noib3 closed 2 months ago

noib3 commented 2 months ago

Using LEB128 seems to bring a 0-10% reduction in message size compared to our own variable-length integer encoding algorithm.

Before:

bincode | Replica: 163.05 KB
bincode | Total insertions: 3.24 MB
bincode | Total deletions: 1.81 MB

serde_json | Replica: 553.96 KB
serde_json | Total insertions: 5.98 MB
serde_json | Total deletions: 3.78 MB

zstd'd bincode | Replica: 134.37 KB
zstd'd bincode | Total insertions: 4.81 MB
zstd'd bincode | Total deletions: 2.47 MB

zstd'd serde_json | Replica: 156.90 KB
zstd'd serde_json | Total insertions: 7.08 MB
zstd'd serde_json | Total deletions: 4.42 MB

After:

bincode | Replica: 160.76 KB
bincode | Total insertions: 3.14 MB
bincode | Total deletions: 1.74 MB

serde_json | Replica: 494.34 KB
serde_json | Total insertions: 5.39 MB
serde_json | Total deletions: 3.43 MB

zstd'd bincode | Replica: 133.02 KB
zstd'd bincode | Total insertions: 4.71 MB
zstd'd bincode | Total deletions: 2.40 MB

zstd'd serde_json | Replica: 146.91 KB
zstd'd serde_json | Total insertions: 6.63 MB
zstd'd serde_json | Total deletions: 4.09 MB