Portable-Network-Archive / liblzma-rs

Bindings to liblzma in Rust (xz streams in Rust)
Apache License 2.0
19 stars 5 forks source link

Unexpected decompression error #134

Open shamilsan opened 3 months ago

shamilsan commented 3 months ago

Hi,

I am faced with an error "Error: premature eof" when trying to decompress a piece of data. I've reproduced this behavior in a test where I try to decompress data using two libraries (liblzma vs lzma-rs). As you can see liblzma produces an error but output has valid decompressed data.

Here it is:

use anyhow::Result;
use liblzma::{
    read::XzDecoder,
    stream::{Filters, Stream},
};
use lzma_rs::decompress::raw::{LzmaDecoder, LzmaParams, LzmaProperties};

fn main() -> Result<()> {
    let input = [
        0_u8, 0, 129, 51, 7, 174, 15, 216, 24, 245, 200, 8, 96, 118, 244, 82, 234, 143, 39, 176,
        35, 215, 18, 32, 29, 41, 69, 26, 118, 10, 168, 24, 71, 93, 49, 151, 59, 85, 71, 218, 240,
        123, 248, 157, 185, 27, 109, 183, 16, 141, 167, 153, 15, 117, 188, 209, 92, 35, 218, 111,
        35, 118, 194, 138, 166, 217, 72, 91, 48, 231, 0, 210, 232, 191, 11, 15, 25, 191, 59, 65,
        86, 86, 188, 62, 76, 254, 10, 58, 169, 117, 77, 10, 238, 182, 202, 120, 14, 50, 254, 171,
        95, 244, 58, 127, 33, 14, 177, 215, 250, 78, 142, 47, 252, 101, 36, 184, 124, 95, 139, 4,
        117, 161, 40, 174, 68, 135, 147, 230, 162, 21, 149, 205, 64, 98, 183, 58, 171, 32, 159,
        223, 17, 252, 144, 232, 40, 190, 116, 0, 95, 219, 147, 77, 67, 214, 212, 60, 173, 95, 201,
        161, 153, 218, 183, 107, 225, 29, 39, 153, 206, 71, 61, 52, 53, 71, 231, 204, 58, 53, 251,
        88, 236, 7, 126, 147, 86, 134, 166, 235, 157, 241, 36, 99, 171, 17, 66, 150, 115, 28, 161,
        207, 53, 76, 38, 201, 19, 176, 185, 243, 30, 34, 229, 191, 199, 106, 105, 33, 209, 170, 8,
        163, 55, 173, 204, 44, 79, 96, 170, 69, 97, 185, 137, 127, 147, 243, 89, 97, 43, 4, 151,
        54, 116, 159, 95, 21, 201, 242, 125, 67, 69, 2, 131, 146, 229, 219, 5, 178, 14, 73, 66, 82,
        126, 246, 173, 8, 222, 36, 164, 20, 199, 159, 227, 132, 71, 35, 217, 203, 64, 156, 9, 145,
        177, 43, 168, 236, 165, 234, 35, 70, 194, 106, 208, 77, 122, 160, 250, 198, 108, 134, 216,
        253, 22, 168, 120, 156, 63, 205, 230, 170, 172, 231, 11, 148, 134, 32, 216, 121, 182, 86,
        96, 86, 151, 110, 92, 87, 17, 96,
    ];

    let props = [93_u8, 0, 0, 16, 0];

    // The `liblzma` part
    let mut filters = Filters::new();
    filters.lzma1_properties(&props)?;

    let stream = Stream::new_raw_decoder(&filters)?;
    let mut output1 = Vec::new();
    let mut decoder = XzDecoder::new_stream(&input[..], stream);
    let res = std::io::copy(&mut decoder, &mut output1);

    // Error: premature eof
    assert!(res.is_err());

    // The `lzma-rs` part
    let d = props[0] as u32;
    let (pb, d) = (d / 45, d % 45);
    let (lp, lc) = (d / 9, d % 9);
    let dict_size = u32::from_le_bytes([props[1], props[2], props[3], props[4]]);

    let props = LzmaProperties { lc, lp, pb };
    let params = LzmaParams::new(props, dict_size, None);
    let mut decoder = LzmaDecoder::new(params, None)?;

    let mut input = std::io::Cursor::new(input);
    let mut output2 = Vec::new();
    decoder.decompress(&mut input, &mut output2)?;

    // Outputs are the same
    assert_eq!(output1, output2);

    Ok(())
}
ChanTsune commented 3 months ago

Hello @shamilsan!

Thank you for your report. As I checked, test cases that are expected to fail with liblzma (original xz-utils) do not fail with lzma-rs. In other words, there is a problem with original xz-utils, or lzma-rs provides functionality that is incompatible with original xz-utils. liblzma is a crate that provides rust-binding for original xz-utils, so there are no plans to directly fix this issue.