Error on build: "Invalid filesystem option set: ^metadata_csum,^huge_file" with docker build

[guysoft]

Hey, I am trying to build pi-gen with the docker build and without it I get the following error. There is no image at the end in the deploy folder. Using Ubuntu 17.04:

I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0xb2a851a7  

Device                                                                      Boot 
Start     End Sectors  Size Id Type
/pi-gen/work/2017-07-13-Raspbian/export-image/2017-07-13-Raspbian-lite.img1      
 8192   94207   86016   42M  c W95 FAT32 (LBA)
/pi-gen/work/2017-07-13-Raspbian/export-image/2017-07-13-Raspbian-lite.img2      
94208 3241835 3147628  1.5G 83 Linux

Command (m for help): The partition table has been altered.
Syncing disks.

/boot: offset 4194304, length 44040192
/:     offset 48234496, length 1611585536
mkfs.fat: warning - lowercase labels might not work properly with DOS or Windows
mke2fs 1.42.12 (29-Aug-2014) 
Invalid filesystem option set: ^metadata_csum,^huge_file
Done! Your image(s) should be in deploy/
/home/ubuntu
/var/lib/docker/volumes/66fb28dda4b5110681d778b8eacbb516798772ed703b99341caa9334b
de50762/_data/2017-07-13-Raspbian/export-image

current workaround is to build without docker

[XECDesign]

You'll need have to revert that commit or use newer e2fsprogs.

$ truncate -s 1G test.img
$ mkfs.ext4 -O ^metadata_csum test.img 
mke2fs 1.42.13 (17-May-2015)
Invalid filesystem option set: ^metadata_csum

$ mkfs.ext4 -O ^metadata_csum test.img
mke2fs 1.43.4 (31-Jan-2017)
Discarding device blocks: done                            
Creating filesystem with 262144 4k blocks and 65536 inodes
Filesystem UUID: 1e400e09-73fb-4eba-b3f7-79cfa48303ae
Superblock backups stored on blocks: 
        32768, 98304, 163840, 229376

Allocating group tables: done                            
Writing inode tables: done                            
Creating journal (8192 blocks): done
Writing superblocks and filesystem accounting information: done

[guysoft]

I using your docker image. My system has the new e2fsprogs. The dockerfile is not up to date and breaks the build. The point of docker is to take care if versions. It gets built with build.sh. and breaks with docker.

[XECDesign]

Ah, I can't do much about that. I am leaving the maintenance of the docker part of pi-gen to the people who use it. PR's welcome.

[guysoft]

I guess cc @cfstras then. My stake here is making nightly builds, i'm trying to replicate the way you are building as best as I can.

[XECDesign]

We have a VM, which runs a nightly cronjob to build the image. There's also a postrun script which does basic cleanup and moves the files where we need them.

[guysoft]

What distribution? Does the vm have a provision script?

[rkubes]

Alternatively, if you're okay with doing a quick patch to your local git clone, then you can remove the dependency on the latest e2fsprogs by only flagging to disable metadata_csum if it's potentially set as a default feature. This will allow you to run the build script on multiple environments, and should not require you to revert that commit based on the e2fsprogs version. Although the logic could be further refined to prevent possible false positives.

diff --git a/export-image/prerun.sh b/export-image/prerun.sh
index a639c9c..67da3da 100755
--- a/export-image/prerun.sh
+++ b/export-image/prerun.sh
@@ -50,8 +50,12 @@ ROOT_DEV=$(losetup --show -f -o ${ROOT_OFFSET} --sizelimit ${
 echo "/boot: offset $BOOT_OFFSET, length $BOOT_LENGTH"
 echo "/:     offset $ROOT_OFFSET, length $ROOT_LENGTH"

+ROOT_FEATURES="^huge_file"
+if grep -q "metadata_csum" /etc/mke2fs.conf; then
+    ROOT_FEATURES="^metadata_csum,$ROOT_FEATURES"
+fi
 mkdosfs -n boot -F 32 -v $BOOT_DEV > /dev/null
-mkfs.ext4 -O ^metadata_csum,^huge_file $ROOT_DEV > /dev/null
+mkfs.ext4 -O $ROOT_FEATURES $ROOT_DEV > /dev/null

 mount -v $ROOT_DEV ${ROOTFS_DIR} -t ext4
 mkdir -p ${ROOTFS_DIR}/boot

[guysoft]

I want to build something that mimcs the Rpi foundation builds. The idea is to have unofficial builds that would be part of the OctoPi project nighties. And like that we will be able to catch changes like the csum. So patching is the opposite of what I want to do.

[XECDesign]

@rkubes That looks sensible. I'd accept that as a PR.

@guysoft The full setup is something like this: Debian Stretch VM, runs a script at midnight which does a quick sanity check of the environment, gets the latest commit from an internal git repo, does the build in a screen session with logging enabled and then moves the files to a place they can be downloaded for internal testing. There is a separate CI system which checks that the basics work on all pi models.

[guysoft]

Ok, I am building nighties now using this: https://github.com/guysoft/pi-gen-nightly

They are uploaded here: http://unofficialpi.org/Distros/raspbian-nightly/

Technically I don't really mind the docker build isn't working. However it is a part in this repo that is broke, so not closing.

[cfstras]

I'll look at getting newer e2fs tools. Need to build a new round of inages soon anyway.

Would someone like to PR a vagrantfile for this repo? Having "the official" way supported here would be good to have in the pursuit of reproducibility.

[cfstras]

okay, look at #87.

@XECDesign: Is there a (formal) description of the build system that produces the official images on https://www.raspberrypi.org/downloads/raspbian/ ?

[cfstras]

That was quick. :tada:

[XECDesign]

There isn't anything formal and everything is subject to change, so it would probably be out of date within a few weeks.

[jahidhasanlinix]

Is there any idea why I am getting this error as below:

$ mke2fs ext4.img -F -E encoding=3 -O casefold 
mke2fs 1.44.1 (24-Mar-2018) 
Invalid filesystem option set: casefold 

[XECDesign]

For general support questions, I'd recommend the forum.

But in this case, the issue is that you're using an option that's not supported by the version of e2fsprogs you're running. I think you'd need at least v1.46.0.

[jahidhasanlinix]

For general support questions, I'd recommend the forum.

But in this case, the issue is that you're using an option that's not supported by the version of e2fsprogs you're running. I think you'd need at least v1.46.0.

How can I update or install v1.46.0, any suggestion please. @XECDesign