tafia / quick-protobuf

A rust implementation of protobuf parser
MIT License
446 stars 82 forks source link

Why does length of encoded data is at the first of bytes? #214

Open farbodpm opened 2 years ago

farbodpm commented 2 years ago

Hi, I was validating quick prot with prost protocol. Encoded data was different from the prost output. Then I realized that the length of the encoded data is added at first. WHY?

nerdrew commented 2 years ago

You can encode it with or without a length prefix.

Here's how to encode / decode without a length prefix:

let mut buf = Vec::new();
let mut writer = Writer::new(&mut buf);
MyMessage::default().write_message(&mut writer)?;

let mut reader = BytesReader::from_bytes(&buf);
let proto = MyMessage::from_reader(&mut reader, &buf)?;

FWIW this also tripped me up when I first started with this library. Suggestions on how to make the docs clearer?

This is a similar issue to #202

TeemuKoivisto commented 1 year ago

Thanks @nerdrew for the tip! Incase anyone else is doing the same, I'm using u8 as the header byte so for me it looks like:

    let mut out = Vec::new();
    let mut writer = Writer::new(&mut out);
    writer.write_u8(header.try_into().unwrap()).unwrap();
    payload.write_message(&mut writer).expect("Unable to serialize message!");
    Message::Binary(out)

For the suggestions, I think adding examples would probably be the most helpful in the docs and maybe comments. In the first place, it of course would be nice to not to need such things. But there's probably a good reason why having to encode length is necessary.

makapuf commented 10 months ago

Agreeing with the fact that this is not clear in the documentation (also would be nice to have the reason for it)