yoshidan / google-cloud-rust

Google Cloud Client Libraries for Rust.
MIT License
233 stars 85 forks source link

Support for Bytes instead of Vec<u8> #96

Closed dezyh closed 1 year ago

dezyh commented 1 year ago

This is a rough example of how prost can be used to generate Bytes instead of Vec<u8> to allow for zero-copy deserialization.

My goal is to have support for both Vec<u8> and Bytes, either generating code for both and exporting separate modules, switching the module based on a feature flag, or something else.

As of now, I've only updated tests for the pubsub crate, as that is my direct use case. I have not tested if the other crates still compile as this is just an example to gauge interest and seek any advice you have.

There is also quite a few new additions to the googleapis protos, which add a bit of noise to the changes in the generated rust code, but here is an example of the difference

dezyh commented 1 year ago

Summary

Added a google-cloud-googleapis/bytes feature, which generates the new Bytes version in src/bytes in addition to the current Vec<u8> version in src. Also, when enabled, the bytes feature will also replace the exported pubsub module with the new Bytes version. Does not currently replace any other exported modules.

Added a google-cloud-pubsub/bytes feature, which just enables the google-cloud-googleapis/bytes feature (mentioned above).

The tests for the google-cloud-pubsub crate all pass, both with & without the new bytes feature. I did stop running the doc test though (still tests that they compile).

Other

Both versions of generated code should be committed into the repo so that they can use the bytes feature without having to generate too. This duplication is slightly annoying, but I think it's the only way.

If you wanted, you could add another cargo test with features=bytes in CI, but I don't think it's too important as the google-cloud-pubsub crate currently doesn't have any code that specifically depends on Vec<u8>/Bytes. The tests did, but .into() makes it easy to support both Vec<u8> and Bytes.

yoshidan commented 1 year ago

Thanks!