squashfs test fails on arm

[sudipm-mukherjee]

I have packaged genimage for Debian and the tests on armhf are failing with squashfs. The following is from the logs:

expecting success: 
    run_genimage squashfs.config test.squashfs &&
    check_size_range images/test.squashfs 4000 4100 &&
    unsquashfs -ls images/test.squashfs  | sed -n '/squashfs-root/s;squashfs-root/;;p' | sort > '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/file-list.test' &&
    check_filelist

1+0 records in
1+0 records out
1048576 bytes (1.0 MB, 1.0 MiB) copied, 0.00337176 s, 311 MB/s
INFO: cmd: "mkdir -p "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp"" (stderr+stdout):
INFO: cmd: "rm -rf "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp"/*" (stderr+stdout):
INFO: cmd: "mkdir -p "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp"" (stderr+stdout):
INFO: cmd: "cp -a "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/root" "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp/root"" (stderr+stdout):
INFO: cmd: "find '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp/root' -depth -type d -printf '%P\0' | xargs -0 -I {} touch -r '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/root/{}' '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp/root/{}'" (stderr+stdout):
INFO: cmd: "mkdir -p "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/images"" (stderr+stdout):
INFO: squashfs(test.squashfs): cmd: "mksquashfs '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp/root' '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/images/test.squashfs' -b 4096 -noappend -comp lzo " (stderr+stdout):
FATAL ERROR: Failed to create thread
INFO: squashfs(test.squashfs): cmd: "rm -f "/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/images/test.squashfs"" (stderr+stdout):
ERROR: squashfs(test.squashfs): failed to generate test.squashfs
not ok 20 - squashfs
#   
#       run_genimage squashfs.config test.squashfs &&
#       check_size_range images/test.squashfs 4000 4100 &&
#       unsquashfs -ls images/test.squashfs  | sed -n '/squashfs-root/s;squashfs-root/;;p' | sort > '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/file-list.test' &&
#       check_filelist
#

The full log can be seen at: https://ci.debian.net/data/autopkgtest/testing/armhf/g/genimage/23179583/log.gz

I understand this may not be a problem with genimage but when I login to an armhf machine and try mksquashfs manually it works.

[michaelolbrich]

INFO: squashfs(test.squashfs): cmd: "mksquashfs '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp/root' '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/images/test.squashfs' -b 4096 -noappend -comp lzo " (stderr+stdout): FATAL ERROR: Failed to create thread

This is the cause. So mksquashfs fails for some reason. The only similar report that I found was caused by out-of-memory problems. Maybe something similar? Can you experiment with -processors x or -mem yM? Add those as extraargs in the squashfs section in test/squashfs.config.

[sudipm-mukherjee]

On Mon, Jul 4, 2022 at 10:43 AM Michael Olbrich @.***> wrote:

INFO: squashfs(test.squashfs): cmd: "mksquashfs '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/tmp/root' '/tmp/autopkgtest-lxc.wzjf46za/downtmp/autopkgtest_tmp/trash directory.basic-images.test/images/test.squashfs' -b 4096 -noappend -comp lzo " (stderr+stdout): FATAL ERROR: Failed to create thread

This is the cause. So mksquashfs fails for some reason. The only similar report that I found was caused by out-of-memory problems. Maybe something similar? Can you experiment with -processors x or -mem yM? Add those as extraargs in the squashfs section in test/squashfs.config.

Thanks Michael. Looks like this problem with armhf is only seen on the autopkgtest servers (which are using lxc based containers) and I could not reproduce the issue on the armhf porterbox I have access to. I am trying to setup similar environment locally to test, and will report back. There are some other failures on s390x and ppc64el also, but I will try to reproduce those failures first before opening a new issue.

-- Regards Sudip

[sudipm-mukherjee]

I am closing this issue as I have not been able to reproduce this. Its only seen on Debian autopkgtest not seen when tried on a porterbox.