onekey-sec / unblob

Extract files from any kind of container formats
https://unblob.org
Other
2.09k stars 80 forks source link

Fix frame_content_size retrieval logic #882

Closed ivanovanton closed 1 week ago

ivanovanton commented 2 weeks ago

According to documentation Frame_Content_Size_flag is a 2 bits value, not 1 bit as the current logic has.

qkaiser commented 2 weeks ago

Good catch ! How did you notice it ? It would be cool to have a ZSTD sample that triggers that part of the code, I'll see what I can do but if you have something you can upload it here and I'll put it at the right location in our integration tests.

ivanovanton commented 2 weeks ago

@qkaiser You just need to provide a file which decompressed size is long enough (> 65791), so FCS_Field_Size is 4 or 8, and Flag_Value is 2 (0b10) or 3 (0b11): https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#frame_content_size

tests.zip 1.zst - decompressed size 65791 (passing) 2.zst - decompressed size 65792 (failing)

qkaiser commented 1 week ago

@ivanovanton git LFS gets in the way of me modifying your forked branch. Could you do this ?

cp 2.zst unblob/tests/integration/compression/zstd/__input__/large_fcs.zstd
cd unblob
poetry run unblob -vvv -e tests/integration/compression/zstd/__output__/ -f -k tests/integration/compression/zstd/__input__/large_fcs.zstd
git add tests/integration/compression/zstd/__input__/large_fcs.zstd
git add tests/integration/compression/zstd/__output__/large_fcs.zstd_extract/zstd.uncompressed
git commit --fixup HEAD
git rebase -i --autosquash HEAD~2
git commit --amend -m 'fix(zstd): fix frame_content_size retrieval logic'

Then push your changes :)

ivanovanton commented 1 week ago

@qkaiser hey, looks good?