search

AgentD / squashfs-tools-ng

A new set of tools and libraries for working with SquashFS images
Other
194 stars 30 forks source link

Broken compressor options in SquashFS images for ZSTD #51

Closed AgentD closed 4 years ago

AgentD commented 4 years ago

@AgentD Here is an example of what I did:

# tar LOT_OF_OPTIONS | tar2sqfs -qf -c zstd /tmp/system.squashfs
# unsquashfs -s /tmp/system.squashfs
Found a valid SQUASHFS 4:0 superblock on /tmp/system.squashfs.
Creation or last append time Thu Jan  1 01:00:00 1970
Filesystem size 309109010 bytes (301864.27 Kbytes / 294.79 Mbytes)
Compression zstd
zstd: bad compression level in compression options structure
zstd: error reading stored compressor options from filesystem!
Block size 131072
Filesystem is not exportable via NFS
Inodes are compressed
Data is compressed
Uids/Gids (Id table) are compressed
Fragments are compressed
Always-use-fragments option is specified
Xattrs are compressed
Duplicates are not removed
Number of fragments 3393
Number of inodes 32640
Number of ids 17

And using gzip:

Found a valid SQUASHFS 4:0 superblock on /tmp/system.squashfs.
Creation or last append time Thu Jan  1 01:00:00 1970
Filesystem size 297931334 bytes (290948.57 Kbytes / 284.13 Mbytes)
Compression gzip
Block size 131072
Filesystem is not exportable via NFS
Inodes are compressed
Data is compressed
Uids/Gids (Id table) are compressed
Fragments are compressed
Always-use-fragments option is specified
Xattrs are compressed
Duplicates are not removed
Number of fragments 3393
Number of inodes 32640
Number of ids 17

The summary is:

Data bytes read: 859.1M
Data bytes written: 294.1M
Data compression ratio: 34%

Data blocks written: 7205
Out of which where fragment blocks: 3393
Duplicate blocks omitted: 139
Sparse blocks omitted: 11

Fragments actually written: 24685
Duplicated fragments omitted: 1612
Total number of inodes: 32640
Number of unique group/user IDs: 17

Maybe the unsquashfs tool is buggy or I did something wrong ;-)

unsquashfs version 4.4 (2019/08/29)
tar2sqfs (squashfs-tools-ng) 0.9.1

Both taken from debian packges. I haven't checked if they patched it thought.

Originally posted by @renard in https://github.com/AgentD/squashfs-tools-ng/issues/4#issuecomment-631504227

AgentD commented 4 years ago

As it turns out, the internal compression level field in the zstd compressor was left uninitialized (i.e. set to zero because the structure is allocated through calloc).

When writing the compressor options, the libsquashfs would happily do so, because they are different from the default value (15). Later, the unsquashfs tool will try to parse those options, but complains that they are invalid, because the default level for zstd compression is 1.

This should be fixed by commit b9b3b7663dffb83e112b5a4aaf786fcab23182fa.