djkoloski / rust_serialization_benchmark

Benchmarks for rust serialization frameworks
514 stars 48 forks source link

Corrections to prost benchmark #62

Closed mumbleskates closed 2 months ago

mumbleskates commented 2 months ago

The prost benchmark is currently discarding the Log::address field, causing it to appear to have significantly higher performance and better compactness than it should.

Before:

log/prost/deserialize                                       1.00  1558.1±16.12µs
log/prost/serialize (encode)                                1.00    235.3±7.06µs
log/prost/serialize (populate + encode)                     1.00   1076.9±3.38µs

log/prost/size 764951
log/prost/zlib 268137
log/prost/zstd 227947

After:

log/prost/deserialize                                       1.00  1941.7±22.57µs
log/prost/serialize (encode)                                1.00    465.8±3.36µs
log/prost/serialize (populate + encode)                     1.00  1248.0±17.10µs

log/prost/size 884628
log/prost/zlib 363130
log/prost/zstd 315494

I also added guardrails to the other benches as well, but this is the main one that it appeared to be performing far better than possible. It is successfully round-tripping values in the other benches.

djkoloski commented 2 months ago

Looks like there's a new issue with nightlies related to aHash 0.7. I'll try to update bson-rust to the latest version since that would eliminate the old dependency.

mumbleskates commented 2 months ago

I can't get an unrestricted set of features to work no matter what i install, flatbuffer generated files are totally wrong with the tool i get in ubuntu 22.04 & 23.10. that's a separate problem though

djkoloski commented 2 months ago

What issues are you seeing? Are they related to the generated Rust files?

mumbleskates commented 2 months ago

i get at least 11 errors like

error[E0405]: cannot find trait `SafeSliceAccess` in crate `flatbuffers`
  --> src/datasets/log/log_generated.rs:42:19
   |
42 | impl flatbuffers::SafeSliceAccess for Address {}
   |                   ^^^^^^^^^^^^^^^ not found in `flatbuffers`

probable issue:

$ flatc --version
flatc version 2.0.8

...but it's kind of a big workflow pain to need to go get and install the latest one to run this, especially considering there are already generated files checked in that it just wants to stomp.

the repo won't build until i install both capnp and flatc but all those seem to do is mess up their respective already-checked-in code.

djkoloski commented 2 months ago

Yeah that's been a problem for a while. As an immediate-term fix you could just disable the build.rs code generation. But for a longer-term fix, I think we'd want to pull down the latest versions of flatc/protoc/capnp from their respective GitHub repos. I already do this manually for Windows because I have no choice.

mumbleskates commented 2 months ago

here's a mini-kludge for it that should work: https://github.com/djkoloski/rust_serialization_benchmark/pull/63