search

dropbox / pb-jelly

A protobuf code generation framework for the Rust language developed at Dropbox.
Apache License 2.0
611 stars 25 forks source link

Zero Copy Deserialization with bytes::Bytes + Cleanup #15

Closed ParkMyCar closed 4 years ago

ParkMyCar commented 4 years ago

This change introduces zero copy de-serialization powered by bytes::Bytes instead of our proprietary blob::Blob. I added a new Rust proto extension, [(rust.zero_copy)] for bytes fields, and updated the codegen so in Rust these fields are ::pb::Lazy<::bytes::Blob>.

I also added a benchmark to test the zero copy de-serialization, versus copying into a Vec<u8>. As such I updated the pb-test crate to use pb-gen. Running the benchmarks on my 2013 MacBook Pro, give me the following results:

test bench::tests::bench_deserialize_vec_bytes       ... bench:   4,952,187 ns/iter (+/- 424,504)
test bench::tests::bench_deserialize_zero_copy_bytes ... bench:         106 ns/iter (+/- 10)
ParkMyCar commented 4 years ago

Also made a number of changes by adding some READMEs to the individual crates, and adding to the repo's README

nipunn1313 commented 4 years ago

looks good!

I think it may be good to rename pb::blob::Blob to something else so that we can disambiguate from the proprietary blob::Blob - though it doesn't feel super critical. Maybe something like PbBuffer and PbBufferReader

In any case - feel free to merge!

ParkMyCar commented 4 years ago

Thanks for the review @nipunn1313! I made the rename from Blob to PbBuffer, that pr is #25