Error: Failed to create overlay: invalid argument

[xsoalokinx]

Hi, getting an error when trying to build a gentoo image here. Using Kernel 5.4.97.

zgrep CONFIG_OVERLAY_FS /proc/config.gz 
CONFIG_OVERLAY_FS=m
# CONFIG_OVERLAY_FS_REDIRECT_DIR is not set
CONFIG_OVERLAY_FS_REDIRECT_ALWAYS_FOLLOW=y
# CONFIG_OVERLAY_FS_INDEX is not set
# CONFIG_OVERLAY_FS_XINO_AUTO is not set
# CONFIG_OVERLAY_FS_METACOPY is not set

neo /tmp/gentoo-lxd # distrobuilder build-lxd gentoo.yaml
+ mv /usr/bin/gpg2 /usr/bin/gpg2.real
+ echo '#!/bin/sh'
+ echo 'exec /usr/bin/gpg2.real --keyserver keyserver.ubuntu.com "$@"'
+ chmod +x /usr/bin/gpg2
+ cp /usr/share/portage/config/repos.conf /usr/share/portage/config/repos.conf.orig
+ sed -i s#hkps://keys.gentoo.org#keyserver.ubuntu.com#g /usr/share/portage/config/repos.conf
+ sed -ri 's#(sync-uri =) .+#\1 rsync://rsync.ca.gentoo.org/gentoo-portage/#' /usr/share/portage/config/repos.conf
+ '[' -e /etc/inittab ']'
+ sed -i 's/^c[0-9]:/#\0/' /etc/inittab
+ sed -i 's/^#\(x1:.*\)/\1/' etc/inittab
+ echo pf:12345:powerwait:/sbin/halt
+ mv /usr/bin/gpg2 /usr/bin/gpg2.real
+ echo '#!/bin/sh'
+ echo 'exec /usr/bin/gpg2.real --keyserver keyserver.ubuntu.com "$@"'
+ chmod +x /usr/bin/gpg2
+ cp /usr/share/portage/config/repos.conf /usr/share/portage/config/repos.conf.orig
+ sed -i s#hkps://keys.gentoo.org#keyserver.ubuntu.com#g /usr/share/portage/config/repos.conf
+ sed -ri 's#(sync-uri =) .+#\1 rsync://rsync.ca.gentoo.org/gentoo-portage/#' /usr/share/portage/config/repos.conf
++ uname -m
+ TARGET=x86_64
+ case "${TARGET}" in
+ TARGET=amd64
+ ln -s net.lo /etc/init.d/net.eth0
+ mkdir -p /etc/runlevels/default
+ rc-update add net.eth0 default
 * service net.eth0 added to runlevel default
+ rm /usr/bin/gpg2
+ mv /usr/bin/gpg2.real /usr/bin/gpg2
+ rm /usr/share/portage/config/repos.conf
+ mv /usr/share/portage/config/repos.conf.orig /usr/share/portage/config/repos.conf
+ '[' -f /usr/bin/gpg2.real ']'
+ '[' -f /usr/share/portage/config/repos.conf.orig ']'
Error: Failed to create overlay: invalid argument

[stgraber]

Anything useful in dmesg?

[xsoalokinx]

Anything useful in dmesg?

actually there is.. not sure how I didnt catch this, i was tailing my logs, but somehow this evaded me.

[420956.885431] overlayfs: filesystem on '/var/cache/distrobuilder.509655173/upper' not supported as upperdir

[stgraber]

What's the filesystem used for that path?

[xsoalokinx]

What's the filesystem used for that path?

zfs version 2.0.3

overlay=on as well, for this path, as it is getting inherited.

[stgraber]

That'd be the issue, you can't use overlayfs on top of zfs, it's incompatible.

[xsoalokinx]

That'd be the issue, you can't use overlayfs on top of zfs, it's incompatible.

Thank you for the insight and for your time!

[stgraber]

Not exactly sure why, I believe it's got to do with the way inode number are allocated, but zfs and overlay just don't mix. On some older kernels, they work but then you get corruption, on newer kernels, there are checks in place to prevent it.

[xsoalokinx]

Not exactly sure why, I believe it's got to do with the way inode number are allocated, but zfs and overlay just don't mix. On some older kernels, they work but then you get corruption, on newer kernels, there are checks in place to prevent it.

Ah.. I am not going to pretend to understand much technical stuff here, im just a tinkerer. It appears to be working now after using --cache-dir and pointing to non-zfs filesystem. Thanks again!

[xsoalokinx]

@stgraber there is one last thing I was wondering about regarding distrobuilder that I can't seem to find anywhere. how does one go about using a specific stage3 tarball while using build-lxd option? for example a hardened one.