3Hren / msgpack-rust

MessagePack implementation for Rust / msgpack.org[Rust]
MIT License
1.1k stars 124 forks source link

Confusion on how to read raw bytes #313

Open jimmiebfulton opened 1 year ago

jimmiebfulton commented 1 year ago

I'm creating a low-level, codec used by multiple languages using MsgPack.

RMP has this encode API:

rmp::encode::write_bin_len(...);
rmp::endoce::write_bin(...);

However, it has this decode API:

rmp::decode::read_bin_len(...);

I was expecting a symmetric rmp::decode::read_bin(...).

What's the idiomatic way to read raw bytes using the API?

dnaka91 commented 1 year ago

After you read the length with read_bin_len, I think all you have to do is reading the data directly from your data input.

For example, assuming you read from some impl std::io::Read:

fn read_bin_to_vec(r: impl Read) -> Result<Vec<u8>> {
    // First read the length of the following bin data
    let len = rmp::decode::read_bin_len(&mut r)? as usize;

    // Then allocate a vec with the read length and
    // read the data into it
    let mut buf = vec![0; len];
    r.read_exact(&mut buf)?;

    Ok(buf)
}

I guess the reason for the missing read_bin is, that there are simply many ways of reading raw data, depending on your data input and the place you want to put the data.

Maybe you want to read into a fresh Vec<u8>, but maybe instead you just want to take a slice out of your input (in case of an already read vector with the raw msgpack data). Or maybe even directly write the bin data out into a file, and, and, and ...