search

djkoloski / rust_serialization_benchmark

Benchmarks for rust serialization frameworks
527 stars 48 forks source link

Add support for BARE (serde_bare) #27

Closed reivilibre closed 2 years ago

reivilibre commented 2 years ago

Thanks for this benchmark!

I like BARE's dense encoding quite a bit, so thought it'd be interesting to see how it stacks up against the competition. I've only run the benchmark for a few of the libraries that I've been considering (partially because some of the other benchmarks don't compile, or need external tools which weren't easy to figure out how to install).

EDIT: Here are the results for the formats I was particularly interested in. BARE seems to have an advantage when it comes to size, which is interesting to know :).

log

This data set is composed of HTTP request logs that are small and contain many strings.

Raw Data

For operations, time per iteration; for size, bytes. Lower is better.

Serialize / deserialize speed and size

Format / Lib Serialize Deserialize Size Zlib Zstd
bare 614.03 us 2.9164 ms 765778 312739 264630
bincode 474.82 us 2.2372 ms 1045784 374305 311761
cbor 1.6595 ms 5.9777 ms 1407835 407372 324081
rkyv 183.76 us 1.3184 ms* 1.9008 ms* 1011488 392809 331932

Zero-copy deserialization speed

Format / Lib Access Read Update
rkyv 1.0035 ns* 591.06 us* 9.5201 us* 600.61 us* 44.896 us

Comparison

Relative to best. Higher is better.

Serialize / deserialize speed and size

Format / Lib Serialize Deserialize Size Zlib Zstd
bare 29.93% 45.21% 100.00% 100.00% 100.00%
bincode 38.70% 58.93% 73.23% 83.55% 84.88%
cbor 11.07% 22.06% 54.39% 76.77% 81.66%
rkyv 100.00% 100.00%* 69.36%* 75.71% 79.62% 79.72%

Zero-copy deserialization speed

Format / Lib Access Read Update
rkyv 100.00%* 0.00%* 100.00%* 1.59%* 100.00%

mesh

This data set is a single mesh. The mesh contains an array of triangles, each of which has three vertices and a normal vector.

Raw Data

For operations, time per iteration; for size, bytes. Lower is better.

Serialize / deserialize speed and size

Format / Lib Serialize Deserialize Size Zlib Zstd
bare 4.1581 ms 7.6070 ms 6000003 5380817 5345900
bincode 4.9309 ms 3.3888 ms 6000008 5380823 5345890
cbor 36.111 ms 59.945 ms 13122324 7527423 6759658
rkyv 474.21 us 329.20 us* 318.65 us* 6000008 5380822 5345892

Zero-copy deserialization speed

Format / Lib Access Read Update
rkyv 1.0050 ns* 12.374 ns* 35.804 us* 34.760 us* 192.25 us

Comparison

Relative to best. Higher is better.

Serialize / deserialize speed and size

Format / Lib Serialize Deserialize Size Zlib Zstd
bare 11.40% 4.19% 100.00% 100.00% 100.00%
bincode 9.62% 9.40% 100.00% 100.00% 100.00%
cbor 1.31% 0.53% 45.72% 71.48% 79.09%
rkyv 100.00% 96.80%* 100.00%* 100.00% 100.00% 100.00%

Zero-copy deserialization speed

Format / Lib Access Read Update
rkyv 100.00%* 8.12%* 97.08%* 100.00%* 100.00%

minecraft_savedata

This data set is composed of Minecraft player saves that contain highly structured data.

Raw Data

For operations, time per iteration; for size, bytes. Lower is better.

Serialize / deserialize speed and size

Format / Lib Serialize Deserialize Size Zlib Zstd
bare 659.83 us 2.7228 ms 356311 213270 198488
bincode 473.08 us 1.9039 ms 569975 240897 232423
cbor 1.6809 ms 5.5620 ms 1109821 347812 274526
rkyv 303.12 us 1.1217 ms* 1.6373 ms* 596952 254571 219976

Zero-copy deserialization speed

Format / Lib Access Read Update
rkyv 495.72 us* 255.19 ns* 494.63 us* 2.7913 us

Comparison

Relative to best. Higher is better.

Serialize / deserialize speed and size

Format / Lib Serialize Deserialize Size Zlib Zstd
bare 45.94% 41.20% 100.00% 100.00% 100.00%
bincode 64.07% 58.92% 62.51% 88.53% 85.40%
cbor 18.03% 20.17% 32.11% 61.32% 72.30%
rkyv 100.00% 100.00%* 68.51%* 59.69% 83.78% 90.23%

Zero-copy deserialization speed

Format / Lib Access Read Update
rkyv 100.00%* 100.00%* 0.05%* 100.00%
djkoloski commented 2 years ago

Thanks for the PR, it looks good so far! As long as the benchmarks compile, I can run them and update the README with the latest numbers. It takes a while to run them all so I usually do it on a separate machine with nothing else running in the background. But you're also welcome to run them yourself!

reivilibre commented 2 years ago

Here we go, I've ran through all the benchmarks now, so this should be OK to merge. Very cool setup! :)

djkoloski commented 2 years ago

Looks good, thanks for the time and effort. I’ll get the benchmarks updated with the latest numbers later today. I’ll leave a comment on this issue when they get released.

djkoloski commented 2 years ago

Benchmarks have been updated!