search

AgentD / squashfs-tools-ng

A new set of tools and libraries for working with SquashFS images
Other
194 stars 30 forks source link

sqfs2tar badly handle 4GB files #87

Closed Kioob closed 2 years ago

Kioob commented 2 years ago

Hi,

it seems that sqfs2tar doesn't properly handle files of 4GB or more : the resulting tar archive is corrupted.

To reproduce :

create an image of 4GB

dd if=/dev/urandom of=poc/poc.ibd bs=8k count=524288 conv=fdatasync

create a squashfs archive

gensquashfs foo.sqfs --compressor zstd --comp-extra level=3 -D poc/

convert to tar (and check result thanks to tar2sqfs)

sqfs2tar foo.sqfs | tar2sqfs bar.sqfs --compressor zstd --comp-extra level=3

result with tar2sqfs

input is not a ustar tar archive!

result with tar

tar: Skipping to next header tar: Archive contains '=\203\272#\254U\341h^\342\346\274' where numeric off_t value expected tar: Archive contains '\032\224\306\211\333\006UU[\022,#' where numeric off_t value expected tar: Exiting with failure status due to previous errors

Some additionnal notes :

Kioob commented 2 years ago

(I checked with 1.0.4 and 1.1.2 versions of sqfs2tar)

I've also checked with different kind of compressor, without affecting behavior (just speed of test). Generating squashfs image thanks to mksquashfs also doesn't affect this problem.

AgentD commented 2 years ago

Hi @Kioob,

thanks for reporting this! I managed to reproduce the issue as you described. I can pack the 4G file into a squashfs, which is mountable and contains the file exactly (as verified by diff). And running the image through sqfs2tar produces a tar header with the size field set to 0.

It should be fixed in commit 531d777.

The tar header field for the size has 12 octal digits and should easily hold values up to 64 GB without using any extensions. I tracked the issue down to the printf format specifier used for generating the header. It used "%0*o ", so the value was truncated to an unsigned int (32 bit on LP64 systems like Linux). With "%0*lo", the result is correct, I can unpack the resulting tarball and the files match byte for byte.

I am currently looking into a follow up issue, as tar2sqfs still complains about a premature end-of-file.

update: should be fixed now with 729850c. I'm actually kind of surprised this wasn't caught by Coverity or similar tools.

A fix has been backported and tested to the 1.0.5 branch (see commit 270fac9).

Signed patches are available here: https://infraroot.at/pub/squashfs/patches-1.1.2/ and here: https://infraroot.at/pub/squashfs/patches-1.0.5/

jdelgadowa commented 2 years ago

Hi,

We've tried this fix with Kioob and glady to tell you it works well!

Thanks for your reactivity :-)