rust-lang / flate2-rs

DEFLATE, gzip, and zlib bindings for Rust
https://docs.rs/flate2
Apache License 2.0
891 stars 158 forks source link

error compressing decompressing data #263

Closed albertoortiz closed 3 years ago

albertoortiz commented 3 years ago

**flate2 = "1.0.20"

rust 1.49.0**

The error happens with protobuf messages of size bigger of 123,743 bytes (more or less). When the message is compressed, after decompressed is impossible to build the protobuf message.

These are my functions:

fn compress(as_user_bytes: ProtobufResult<Vec<u8>>) -> Vec<u8> {
    let mut e = GzEncoder::new(Vec::new(), Compression::default());
    e.write(&as_user_bytes.unwrap()[..]);
    let compressed_as = e.finish().unwrap();
    return compressed_as;
}

fn decompress(b: &Vec<u8>) -> Vec<u8> {
    let mut gz = GzDecoder::new(&b[..]);
    let mut s: Vec<u8> = Vec::new();
    gz.read_to_end(&mut s);
    return s;
}

Used like:

              let mut as_user_data: userdata_aerospike::UserData = protobuf::Message::parse_from_bytes(&bytes[..]).unwrap();
                    println!("user from as ok");
                    let bytes = compress(as_user_data.write_to_bytes());
                    let s = decompress(&bytes);
                    let parsed_data = protobuf::Message::parse_from_bytes(&s[..]);
                    let _fail: userdata_aerospike::UserData = parsed_data.unwrap();

There is no problem moving these messages in/out Aerospike without compression. Any idea? Thank you

albertoortiz commented 3 years ago

seems like after adding flate2 = { version="1.0.20", features = ["zlib"], default-features = false} there are no more problems with compressing/decompressing sizes.

Thank you

oyvindln commented 3 years ago

You need to use write_all to guarantee that all input is written, or use the return value from the write call to continue writing until everything is done.

albertoortiz commented 3 years ago

thank you!