search

djkoloski / rust_serialization_benchmark

Benchmarks for rust serialization frameworks
514 stars 48 forks source link

option to use already-existing generated files for flatbuffers and capnp #63

Closed mumbleskates closed 1 month ago

mumbleskates commented 2 months ago

plausibly this could also be done the other direction, making code regeneration for these a feature that has to be enabled, but is disabled by default. that would probably be fair, and might also serve as a guardrail that makes the tests fail when the generated code has rotted.

mumbleskates commented 2 months ago

dang looks like this is true of protoc too but that code isn't checked in. at least that one is older and changes slower

mumbleskates commented 1 month ago

ah dang i thought this was lying cold for a while there. i kinda cleaned this up in my working branch a little more; i have features for "regenerate-capnp", "regenerate-flatbuffers", and "regenerate-prost" that switch the respective codegens on (and committed the prost generated files into the tree to match), so now you can do the whole thing without binaries for protoc, capnp, or flatc at all; meanwhile, CI can still assert that these generated files are up to date

i also added a feature-flag for pprof+flamegraph that works very nicely and easily, at least on linux, that i've been finding useful. and i've added my own encoding library to the bench set, but that doesn't have to be in here until it's cooked a little longer.

you are welcome to any and all of these changes, which i can whip up into coherent pull requests as desired

djkoloski commented 1 month ago

Those improvements would be great! No pressure, PR if you would line. And I'm excited to see how your own library stacks up. 🙂

mumbleskates commented 1 month ago

I'm excited to see how your own library stacks up. 🙂

tl;dr encoding very similar to protobuf but slightly more compact, with distinguished decoding and canonical output. it's a rewrite of prost; at present it's slightly faster, mostly because i went back and tackled the inlining for a few hours since i remade most of the encoding/decoding stack to use trait-based dispatch.

i have a couple more innovations to apply to the encoding but it needs more work. (would you believe that optimization gets very, very strange when your program is writing bytes to a buffer from the end going backwards instead of forwards? 150% speedups from adding .rev() to a couple iterators, glittering off the shoulder of Orion)