Incompatible LZ4 compression

[mscdex]

When selecting LZ4 for the compression and generating the squashfs, during boot the Linux kernel panics because it couldn't load the rootfs. In particular there is a kernel message (seen via wifibox console) saying: "SQUASHFS error: Unknown LZ4 version"

[pgj]

Hi @mscdex, thanks for reporting this issue. I will look into the details and change the Linux kernel configuration accordingly, which will hopefully provide a resolution.

[pgj]

I have investigated the situation and it seems that there is a mismatch between squashfs-tools and the Linux kernel hence this error. I do not still fully understand the reasons but it might be better to simply retire the support for the LZ4 compression. The squashfs-tools documentation suggests that this an experimental feature and may not be usable.

[mscdex]

After a bit of digging it seems to be because squashfs-tools uses the C API for compressing with LZ4, however the LZ4 "legacy" format that the kernel still uses is only available via the LZ4 CLI tool because it utilizes a special, non-public function that still compresses in this "legacy" format. So squashfs-tools is unknowingly compressing with LZ4's newer, non-legacy format. I'm guessing that at some point in the past the "legacy" format was still the default and that fact changed over some time.

Executing lz4cli -l externally/manually should compress it in the "legacy" format that works with the kernel.

[pgj]

Ah, great, thanks for the hints! The lz4cli tool could be then integrated into the port as a workaround for this issue.

[pgj]

I am not sure if you had any exact plans for the fix, but I found a fork called squashfs-tools-ng that can do proper LZ4 compression. For instance, it has the sqfs2tar and tar2sqfs tools which could be used to re-compress the faulty LZ4-compressed squashfs-tools image. Or the build process could be just simply re-based to squashfs-tools-ng completely, but that is a bigger effort as its input interface is different.

[mscdex]

No, I did not have any immediate plans since xz seems to work well enough with the Celeron N5105 I'm using. I still think having working LZ4 would be worth it though for more underpowered machines since it has significantly faster decompression speed compared to the other available compression algorithms.

[pgj]

This is now fixed in f98c558fb4b9e51c92f1c659bbae7c14eab708be -- please test it if you have the time.

[pgj]

@mscdex I have not received any negative feedback from you, and it works for me (as far as I tested). I am now closing this ticket. Feel free to re-open if you still experience any issues.

[mscdex]

Sorry, I'm not able to test. I ended up switching to a different OS.