dropbox / rust-brotli

Brotli compressor and decompressor written in rust that optionally avoids the stdlib
https://dropbox.tech/infrastructure/-broccoli--syncing-faster-by-syncing-less
BSD 3-Clause "New" or "Revised" License
818 stars 83 forks source link

`BrotliDecompress` panics if output writer stops accepting data #213

Closed ctz closed 1 month ago

ctz commented 5 months ago

Reproducer:

use std::io::Cursor;

fn main() {
    // this is a valid compression of [0u8; 2048];
    let compression = [27, 255, 7, 0, 36, 0, 194, 177, 64, 114, 7];
    // output buffer doesn't have enough space
    let mut output_buffer = [0u8; 2047];

    brotli::BrotliDecompress(
        &mut Cursor::new(compression),
        &mut Cursor::new(&mut output_buffer[..]),
    )
    .unwrap_err();
}

This panics for me with:

thread 'main' panicked at /home/jbp/.cargo/registry/src/index.crates.io-6f17d22bba15001f/brotli-decompressor-4.0.0/src/lib.rs:284:13:
assertion `left == right` failed
  left: true
 right: false
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

I think there's a confusion there about what io::Write::write is allowed to return -- Ok(0) is valid, and in this case should be be promoted to an error.

danielrh commented 5 months ago

Is there an analogous bug in the compressor, or did fixing the decompressor also resolve this issue?

ctz commented 5 months ago

Apologies, I put this issue in the wrong repo. Thanks for taking the decompressor fix and making a release.

But for the sake of completeness, here's a test with a similar case on the compressor:

diff --git a/src/enc/test.rs b/src/enc/test.rs
index fff4856..2c56f89 100644
--- a/src/enc/test.rs
+++ b/src/enc/test.rs
@@ -549,6 +549,27 @@ fn test_roundtrip_empty() {
     assert_eq!(output_offset, 0);
     assert_eq!(compressed_offset, compressed.len());
 }
+
+#[cfg(feature="std")]
+#[test]
+fn test_compress_into_short_buffer() {
+    use std::io::{Cursor, Write};
+
+    // this plaintext should compress to 11 bytes
+    let plaintext = [0u8; 2048];
+
+    // but we only provide space for 10
+    let mut output_buffer = [0u8; 10];
+    let mut output_cursor = Cursor::new(&mut output_buffer[..]);
+
+    let mut w = crate::CompressorWriter::new(&mut output_cursor,
+                                         4096, 4, 22);
+    w.write(&plaintext).unwrap_err();
+    w.into_inner();
+
+    println!("{output_buffer:?}");
+}
+
 /*

This test unfortunately does not terminate, as it is making no progress in https://github.com/dropbox/rust-brotli/blob/37d403b437c3cbd1c686871a4167290aab401704/src/enc/writer.rs#L131-L142

And fixing that seems tricky without a breaking change. Ideally a parameter would be added to this function for the ErrType value to return on seeing Ok(0). For std callers that would be std::io::ErrorKind::WriteZero.into().

In our use case we compress into a Cursor over a Vec, so won't be affected by this. However happy to put together a PR that:

danielrh commented 5 months ago

I would really appreciate that PR...

this might warrant a breaking change in the future, but maybe we could do a non-breaking release first with your PR and then add that WriteZero capability into the next major version on its heels?

danielrh commented 1 month ago

I think the recent commit has improved the situation--but you're right it's a breaking change--moved version to 7.0.0