Cannot build 32-bit images (bullseye, bookworm) using standard build.sh on i686 Debian any more

[chrishamm]

I'm having issues building Bullseye- and Bookworm-based 32-bit RPi images using the standard build.sh script on Debian i686 Bullseye and Bookworm. I am not sure why, but the Docker-based build via build-docker.sh succeeds without problems.

On i686 Debian Bullseye/Bookworm I get Allocating guest commpage: Cannot allocate memory with the standard qemu-user-static package in the debootstrap log (as reported here) and with qemu 7abc8cabad (as recommended here) or qemu on Buster, the passwd and/or systemd package causes segfaults. When trying to build Bullseye/Bookworm on i686 Debian Buster, qemu appears to segfault as well, even though building RPi OS Buster still works.

I also tried to build RPi OS Bookworm 32-bit using the latest qemu from GitHub (latest commit from master), but when I do that, apt complains about the RPi package lists:

[13:18:43] Begin /home/vm/pi-gen/stage0/00-configure-apt/00-run.sh
Get:1 http://archive.raspberrypi.com/debian bookworm InRelease [23.7 kB]
Hit:2 http://raspbian.raspberrypi.com/raspbian bookworm InRelease
Err:1 http://archive.raspberrypi.com/debian bookworm InRelease
  At least one invalid signature was encountered.
Err:2 http://raspbian.raspberrypi.com/raspbian bookworm InRelease
  At least one invalid signature was encountered.
Reading package lists... Done
W: GPG error: http://archive.raspberrypi.com/debian bookworm InRelease: At least one invalid signature was encountered.
E: The repository 'http://archive.raspberrypi.com/debian bookworm InRelease' is not signed.
N: Updating from such a repository can't be done securely, and is therefore disabled by default.
N: See apt-secure(8) manpage for repository creation and user configuration details.
W: An error occurred during the signature verification. The repository is not updated and the previous index files will be used. GPG error: http://raspbian.raspberrypi.com/raspbian bookworm InRelease: At least one invalid signature was encountered.

The arm64 images are generated without problems on x86_64 Debian Bookworm with the standard qemu package.

Does the Docker i686 Debian image perhaps use a different qemu version? Is there a qemu build that is known to work with i686/32-bit RPi OS builds?

[XECDesign]

Have you tried building from Debian Bookworm with the version of qemu they ship?

[chrishamm]

Yes, that's the first thing I tried. That provoked Allocating guest commpage: Cannot allocate memory on i686 for 32-bit RPi OS. Interestingly enough I could generate arm64 builds on x86_64 as you'd expect.

PS: Sorry, I just realized I had mixed up Bullseye/Buster with Bullseye/Bookworm in my previous report.

[XECDesign]

Could you try the sysctl workaround I mentioned here? https://gitlab.com/qemu-project/qemu/-/issues/633

[chrishamm]

I just did but it doesn't seem to help:

Setting up passwd (1:4.13+dfsg1-1) ...
Allocating guest commpage: Cannot allocate memory

I reverted to the standard qemu-user-static package, rebooted, and applied your sysctl change before I kicked off a new build.

[chrishamm]

I also tried your sysctl workaround with my fresh qemu build from master but that provoked a different error:

W: Failure trying to run: chroot "/home/vm/pi-gen/work/DuetPi/stage0/rootfs" mount -t sysfs sysfs /sys
W: See /home/vm/pi-gen/work/DuetPi/stage0/rootfs/debootstrap/debootstrap.log for details
W: Failure trying to run: chroot "/home/vm/pi-gen/work/DuetPi/stage0/rootfs" /sbin/ldconfig
W: See /home/vm/pi-gen/work/DuetPi/stage0/rootfs/debootstrap/debootstrap.log for detail

...

qemu-arm: /usr/bin/mount: Unable to find a guest_base to satisfy all guest address mapping requirements
  00000000-00000fff
  00010000-0001b02f
  ffff0000-ffff1eff
qemu-arm: /usr/sbin/ldconfig: Unable to find a guest_base to satisfy all guest address mapping requirements
  00000000-00000fff
  00010000-000c0463
  ffff0000-ffff1eff

[XECDesign]

Okay, I know very little about how qemu works under the hood and getting a minimal testcase together to be able to report the issue upstream in a way that would convince someone to take a look is not an undertaking I'd like to volunteer for any time soon.

Are you building from i686 rather than amd64 for a particular reason? Is that actually what you run and would like to use or is it to workaround another issue qemu had a while back with readdir? I think that's not a problem nowadays, so if an amd64 host works, there's probably no reason to use i686 and it might be time to explicitly state we don't support it.

[chrishamm]

Oh, I see! Yes, the only reason I am still using an i686 VM is because I was aware that some issues could arise when building 32-bit images on 64-bit. If that's no longer a problem, I'll just build Bullseye and Bookworm in my x86_64 VM. In that case please consider removing this statement from the README again :)

Also note that the build-docker script still fetches an i386 container if running on x64. It might be worth changing that as well if you plan to drop i686 support anyway.

I'll check again if the 32-bit build succeeds in my 64-bit VM and if it does, I'll close this issue. Otherwise I need to find a way to let someone reproduce it (using libvirt here).

[chrishamm]

All good, I think. Thanks for your feedback!