AgentD / squashfs-tools-ng

A new set of tools and libraries for working with SquashFS images
Other
194 stars 30 forks source link

Bad start offset for files smaller than one block with DONT_FRAGMENT set #63

Closed showermat closed 3 years ago

showermat commented 3 years ago

Hello,

I'm trying to make SquashFS archives with the constraint that certain files need to be mmapable. Happily, it looks like this is achievable using the "don't compress" and "don't fragment" flags for those files. Since this isn't exposed by gensquashfs, I'm writing my own program using libsquashfs to write the file.

Unfortunately, when I write a file smaller than one block (128 KiB) with SQFS_BLK_DONT_FRAGMENT set, it seems that the offset of the file data from the beginning of the archive is being set to zero, resulting in invalid input to the decompressor (or resulting in the archive appearing to contain a truncated version of itself if SQFS_BLK_DONT_COMPRESS is also set). This behavior is consistent when reading the file through libsquashfs and the kernel, indicating it's probably an issue with the writing.

I've attached a simple example containing one file exhibiting the problem (unfortunately, I had to give it a .txt extension to get it to upload). Please let me know if there's anything I can do to further help debug this issue. Thanks!

AgentD commented 3 years ago

Hi,

I did finally get some time to look into this over the weekend and I think it is a problem with the state machine inside the block processor not setting the "last block" flag in your exact scenario and subsequently not updating the location in the inode.

Sadly the bock processor is the one, fiddly part, so it took a little longer than expected. This patch passes all the tests on my end:

patch.txt

If it works for you as well, I will document and push this. I'm also working on some proper test test cases for the block processor, which should be a lot easier now that the writer and processor have been separated before the 1.0 release.

showermat commented 3 years ago

Indeed, I am unable to reproduce the issue with this patch applied. Thanks so much for taking a look!