kaitai-io / kaitai_struct_formats

Kaitai Struct: library of binary file formats (.ksy)
http://formats.kaitai.io
712 stars 203 forks source link

Squashfs #596

Open tisoft opened 2 years ago

tisoft commented 2 years ago

Added support for the SquashFS filesystem.

Based on the SqashFS Superblock from @abitrolly from #308.

Sample application using this structure: https://github.com/tisoft/jsquashfs

Replaces #308

tisoft commented 2 years ago

Not all inode types are implemented yet, but the most used basic_file, basic_directory and extended_fil) are implemented. The other types shouldn't be a big problem.

Export and xattrs tables are not yet implemented.

SquashFS uses compressed metadata blocks. To fully support the format, it is needed to uncompress those blocks and parse the resulting bytes. Currently a custom decompress function is used for this. This could probably be replaced with kaitai.compress functions.

Unfortunately it is possible for inodes in the inode table to span 2 metablocks. That is why currently a custom concat function is needed, that concatenates the uncompressed bytes from a list of metadata blocks.

This format is currently very memory intensive. The real SquashFS implementations use caching to only have those metablocks in memory, that are currently needed. I couldn't find a good solution for that in kaitai.

The custom compression function also allows to automatically pad the resulting byte array with 0x00. This is used for sparse files in SquashFS. It is only needed in data blocks, so it is not needed for parsing the metadata. It could be removed from the format and could be handled any the calling application. For now I included it, to have a "complete" format.