search

gendx / lzma-rs

An LZMA decoder written in pure Rust
MIT License
126 stars 27 forks source link

XZ: Unknown filter id 7 #94

Open wcampbell0x2a opened 1 year ago

wcampbell0x2a commented 1 year ago

Trying to move my library to using this rust implementation, but running into some issues with decompressing some images. I added a print of the bytes for your help if you wish to fix this issue. It might be something non-standard with these pieces of firmware.

--- test_08 stdout ----
Fetching file out.squashfs ...  => Hash mismatch: found debe0986658b276be78c3836779d20464a03d9ba0a40903e6e8e947e434f4d67, expected ce0bfab79550885cb7ced388caaaa9bd454852bf1f9c34789abc498eb6c74df6
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, ea, fe, 01, 80, 80, 08, 21, 01, 0a, 00, cc, 42, c9, 5a, e1, ff, ff, 7f, 62, 5d, 00, 3f, 91, 45, 84, 68]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 9d, 83, 03, 80, 80, 08, 21, 01, 0a, 00, d2, ff, 95, 0d, e1, ff, ff, c1, 95, 5d, 00, 24, 22, 44, 82, 2f]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d4, 84, 03, 80, 80, 08, 21, 01, 0a, 00, 71, be, 5b, 0b, e1, ff, ff, c2, 4c, 5d, 00, 30, 12, 0c, ac, 03]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d2, a4, 03, 80, 80, 08, 21, 01, 0a, 00, 95, 4c, b4, 84, e1, ff, ff, d2, 4a, 5d, 00, 07, 8b, 1c, 3d, f0]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 9b, d9, 03, 80, 80, 08, 21, 01, 0a, 00, 7d, 44, f7, e7, e1, ff, ff, ec, 93, 5d, 00, 45, 9b, 00, 83, 83]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, e7, 9e, 03, 80, 80, 08, 21, 01, 0a, 00, a5, 4f, 71, b1, e1, ff, ff, cf, 5f, 5d, 00, 24, 31, d4, cf, fd]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 04, c1, cd, ff, 01, 80, 80, 08, 07, 00, 21, 01, 0a, 00, 00, 00, a5, 77, a2, 68, e1, ff, ff, 7f, c5, 5d, 00, 20]
thread 'test_08' panicked at 'called `Result::unwrap()` on an `Err` value: XzError("Unknown filter id 7")', src/compressor.rs:103:71
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

---- test_openwrt_netgear_ex6100v2 stdout ----
File openwrt-22.03.2-ipq40xx-generic-netgear_ex6100v2-squashfs-factory.img has matching hash inside hash list, skipping download
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, a9, 0c, 80, 40, 21, 01, 0c, 00, 00, 00, d8, 5a, ed, b2, e0, 1f, ff, 06, 21, 6c, 00, 01, 80, 1b, e0, 10]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, b6, 0f, 80, 40, 21, 01, 0c, 00, 00, 00, 78, bc, c5, 5d, e0, 1f, ff, 07, ae, 6c, 00, 7f, 80, 3c, 1a, 25]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d2, 0c, 80, 40, 21, 01, 0c, 00, 00, 00, b7, 24, 44, bc, e0, 1f, ff, 06, 4a, 6c, 00, 00, 68, 0a, 10, 2f]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, e0, 08, 80, 40, 21, 01, 0c, 00, 00, 00, be, 27, ae, 8d, e0, 1f, ff, 04, 58, 6c, 00, 00, 68, 82, 3e, fe]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, e1, 0c, 80, 40, 21, 01, 0c, 00, 00, 00, 8c, 1d, 80, 3f, e0, 1f, ff, 06, 59, 6c, 00, 00, 68, 82, 3e, fd]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d4, 07, c0, 21, 21, 01, 0c, 00, 00, 00, 91, 4f, 53, 37, e0, 10, bf, 03, cc, 6c, 00, 52, 00, 3c, 1a, 25]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d4, 07, c0, 21, 21, 01, 0c, 00, 00, 00, 91, 4f, 53, 37, e0, 10, bf, 03, cc, 6c, 00, 52, 00, 3c, 1a, 25]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d1, 20, 80, 40, 21, 01, 0c, 00, 00, 00, 03, ee, b6, 33, e0, 1f, ff, 10, 49, 6c, 00, 1d, 00, 31, 00, 52]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, c5, 01, b0, 03, 21, 01, 0c, 00, 00, 00, e8, 65, 57, 78, e0, 01, af, 00, bd, 6c, 00, 41, 2e, 3c, 40, 0d]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, f1, 08, 80, 40, 21, 01, 0c, 00, 00, 00, 68, cd, c2, 45, e0, 1f, ff, 04, 69, 6c, 00, 1a, 01, bc, 1b, 30]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d1, 20, 80, 40, 21, 01, 0c, 00, 00, 00, 03, ee, b6, 33, e0, 1f, ff, 10, 49, 6c, 00, 1d, 00, 31, 00, 52]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 97, 1b, 80, 40, 21, 01, 0c, 00, 00, 00, eb, c7, 76, 0c, e0, 1f, ff, 0d, 8f, 6c, 00, 00, 04, ff, f2, c3]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, b9, 15, 80, 40, 21, 01, 0c, 00, 00, 00, 1a, 2f, 20, 95, e0, 1f, ff, 0a, b1, 6c, 00, 33, 7f, fc, 40, 00]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, db, 13, e9, 26, 21, 01, 0c, 00, 00, 00, 92, 40, ea, 14, e0, 13, 68, 09, d3, 6c, 00, 00, 68, 9e, 60, af]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 04, c1, a2, 8d, 05, 98, eb, 0f, 07, 00, 21, 01, 0c, 00, 00, 00, d6, 56, 7d, 17, e2, 91, 75, ef, ff, 6c, 00, 11]
thread 'test_openwrt_netgear_ex6100v2' panicked at 'called `Result::unwrap()` on an `Err` value: XzError("Unknown filter id 7")', src/compressor.rs:103:71

---- test_tplink_ax1800 stdout ----
File img-1571203182_vol-ubi_rootfs.ubifs has matching hash inside hash list, skipping download
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, f7, 0e, 80, 40, 21, 01, 02, 00, 00, 00, 96, b3, 19, db, e0, 1f, ff, 07, 6f, 6c, 00, 01, 00, 1b, e0, 10]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 88, 0e, 80, 40, 21, 01, 02, 00, 00, 00, 83, 6d, 5b, dc, e0, 1f, ff, 07, 00, 6c, 00, 00, 68, a3, b3, 14]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, f8, 0f, 80, 40, 21, 01, 02, 00, 00, 00, 58, fc, 69, 3d, e0, 1f, ff, 07, f0, 6c, 00, 17, 9a, 49, c6, 93]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, ef, 11, 80, 40, 21, 01, 02, 00, 00, 00, 2a, ff, 19, 9c, e0, 1f, ff, 08, e7, 6c, 00, 00, 68, 36, 3b, 87]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, b6, 10, 80, 40, 21, 01, 02, 00, 00, 00, eb, 46, 63, ce, e0, 1f, ff, 08, 2e, 6c, 00, 00, 69, 86, 3a, ee]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, ba, 0f, 80, 40, 21, 01, 02, 00, 00, 00, c5, 2b, 26, a7, e0, 1f, ff, 07, b2, 6c, 00, 7f, 80, 3c, 1a, 22]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 84, 10, 80, 40, 21, 01, 02, 00, 00, 00, ee, 14, 65, a2, e0, 1f, ff, 07, fc, 6c, 00, 00, 62, 33, d1, c6]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, e0, 13, 80, 40, 21, 01, 02, 00, 00, 00, 21, 8c, e4, 43, e0, 1f, ff, 09, d8, 6c, 00, 00, 68, 2c, 21, 4b]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, e9, 10, 80, 40, 21, 01, 02, 00, 00, 00, 2e, 9b, 7c, 86, e0, 1f, ff, 08, 61, 6c, 00, 00, 68, 08, b7, 8b]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, a2, 11, 80, 40, 21, 01, 02, 00, 00, 00, 3a, 73, 5d, f7, e0, 1f, ff, 08, 9a, 6c, 00, 01, 80, 1b, e0, 10]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 83, 0c, 80, 40, 21, 01, 02, 00, 00, 00, f2, be, 4d, 0a, e0, 1f, ff, 05, fb, 6c, 00, 00, 67, fe, e0, 12]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, c3, 0c, 80, 40, 21, 01, 02, 00, 00, 00, 52, b9, f7, 94, e0, 1f, ff, 06, 3b, 6c, 00, 00, 67, fe, e0, 12]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d9, 0c, 80, 40, 21, 01, 02, 00, 00, 00, 73, a8, 7b, a4, e0, 1f, ff, 06, 51, 6c, 00, 00, 80, 2e, a3, b5]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, c6, 09, fb, 29, 21, 01, 02, 00, 00, 00, a9, b3, 5d, d7, e0, 14, fa, 04, be, 6c, 00, 05, 80, 30, d4, 90]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, c6, 09, fb, 29, 21, 01, 02, 00, 00, 00, a9, b3, 5d, d7, e0, 14, fa, 04, be, 6c, 00, 05, 80, 30, d4, 90]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 99, 22, 80, 40, 21, 01, 02, 00, 00, 00, e2, f6, f2, 70, e0, 1f, ff, 11, 11, 6c, 00, 00, 6a, 9e, 95, af]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, cd, 07, a0, 15, 21, 01, 02, 00, 00, 00, 7a, b3, f0, 80, e0, 0a, 9f, 03, c5, 6c, 00, 37, 00, 31, 18, 81]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, a8, 0f, 80, 40, 21, 01, 02, 00, 00, 00, 10, 7a, 7d, 84, e0, 1f, ff, 07, a0, 6c, 00, 7f, 05, 14, 42, 80]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 99, 22, 80, 40, 21, 01, 02, 00, 00, 00, e2, f6, f2, 70, e0, 1f, ff, 11, 11, 6c, 00, 00, 6a, 9e, 95, af]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, b5, 1f, 80, 40, 21, 01, 02, 00, 00, 00, 39, 32, ed, fa, e0, 1f, ff, 0f, ad, 6c, 00, 37, 9b, 88, ca, c0]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d9, 1e, 80, 40, 21, 01, 02, 00, 00, 00, 84, c2, 4d, 26, e0, 1f, ff, 0f, 51, 6c, 00, 21, 90, 46, 63, 01]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, bf, 24, 80, 40, 21, 01, 02, 00, 00, 00, ff, fc, ab, 41, e0, 1f, ff, 12, 37, 6c, 00, 04, 80, 09, e7, 03]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 96, 1e, 80, 40, 21, 01, 02, 00, 00, 00, a9, 9e, fc, 49, e0, 1f, ff, 0f, 0e, 6c, 00, 3a, 24, 3e, 61, e0]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, ad, 21, 80, 40, 21, 01, 02, 00, 00, 00, 65, e8, 67, 28, e0, 1f, ff, 10, a5, 6c, 00, 18, 8c, 2c, b4, 00]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 81, 21, 80, 40, 21, 01, 02, 00, 00, 00, 3b, 0b, 06, 7d, e0, 1f, ff, 10, 79, 6c, 00, 37, 19, c9, ea, 30]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, bf, 12, 80, 40, 21, 01, 02, 00, 00, 00, a7, 45, 80, 1c, e0, 1f, ff, 09, 37, 6c, 00, 20, 9d, 4a, 86, f2]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, e7, 0b, d0, 15, 21, 01, 02, 00, 00, 00, e7, 42, eb, 92, e0, 0a, cf, 05, df, 6c, 00, 00, 00, 7f, fe, 90]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, 8b, 2f, 80, 80, 10, 21, 01, 0c, 00, 00, 6d, 72, b3, 7b, e3, ff, ff, 17, 83, 6c, 00, 00, 7f, fe, 97, fe]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 03, c0, d8, 2d, 80, 80, 10, 21, 01, 0c, 00, 00, a0, 67, 66, 07, e3, ff, ff, 16, d0, 6c, 00, 00, 7f, fe, 97, fe]
[fd, 37, 7a, 58, 5a, 00, 00, 01, 69, 22, de, 36, 04, c1, d0, e0, 06, 80, 80, 10, 07, 00, 21, 01, 0c, 00, 00, 00, 47, 1e, 76, 6c, e2, 45, 63, ef, fe, 6c, 00, 3f]
thread 'test_tplink_ax1800' panicked at 'called `Result::unwrap()` on an `Err` value: XzError("Unknown filter id 7")', src/compressor.rs:103:71

issue: https://github.com/wcampbell0x2a/backhand/issues/95 branch: https://github.com/wcampbell0x2a/backhand/tree/use-lzma-rs

gendx commented 1 year ago

This filter is not yet implemented. For the moment, only the LZMA2 filter is implemented within the XZ container format: https://github.com/gendx/lzma-rs/blob/v0.3.0/src/decode/xz.rs#L173-L183.

gendx commented 1 year ago

Looking at the tukaani project, this appears to be a filter specific to ARM: https://github.com/tukaani-project/xz/blob/v5.4/src/liblzma/api/lzma/bcj.h#L41.