search

stepancheg / rust-protobuf

Rust implementation of Google protocol buffers
MIT License
2.81k stars 382 forks source link

Delta-encoding/decoding helpers #615

Open nyurik opened 2 years ago

nyurik commented 2 years ago

Delta-encoding seems to be a common PBF-encoding pattern, where a repeated field is encoded as delta from the previous value rather than absolute values. Each user of the lib has to implement the same logic. Would it be possible to add encoder/decoder for such cases?

Simple case - an iterator of i64 that reads Vec<i64>, e.g. this read implementation.

For encoding, perhaps an iterator that encodes values as they pass through, so it can be converted to a vector with collect()? Thx!

stepancheg commented 2 years ago

What is has to do with rust-protobuf?

rust-protobuf implements Google Protocol Buffers, which is concrete protocol and specification, and it does not mention delta encoding.

nyurik commented 2 years ago

@stepancheg you are right - it is not part of the spec. My thinking is that since this is a fairly common pattern for protobufs, it might be good to include it in the lib - even though technically it might be as a separate crate -- a tiny one at that.