Closed nullromo closed 3 months ago
I was able to change my CI script
- ./build.sh -c config
+ ./build.sh -c config || cat work/Dicon-Raspbian/stage0/debootstrap.log
I got the following for the log printout:
Nothing appears to be wrong.
Normally, the last thing debootstrap does is remove the directory. In the past, there can been cases where it exits successfully even after a part of debootstrap fails, so pi-gen makes sure the directory is gone before proceeding.
The problem happens here:
W: Failure trying to run: chroot "/builds/kkovacs/oss_97542/pi-gen/work/Dicon-Raspbian/stage0/rootfs" dpkg --force-overwrite --force-confold --skip-same-version --install /var/cache/apt/archives/dmsetup_2%3a1.02.185-2_armhf.deb /var/cache/apt/archives/libapparmor1_3.0.8-3_armhf.deb /var/cache/apt/archives/libargon2-1_0~20171227-0.3+deb12u1_armhf.deb /var/cache/apt/archives/libcryptsetup12_2%3a2.6.1-4~deb12u2_armhf.deb /var/cache/apt/archives/libdevmapper1.02.1_2%3a1.02.185-2_armhf.deb /var/cache/apt/archives/libfdisk1_2.38.1-5+deb12u1_armhf.deb /var/cache/apt/archives/libffi8_3.4.4-1_armhf.deb /var/cache/apt/archives/libip4tc2_1.8.9-2_armhf.deb /var/cache/apt/archives/libjson-c5_0.16-2_armhf.deb /var/cache/apt/archives/libkmod2_30+20221128-1_armhf.deb /var/cache/apt/archives/libp11-kit0_0.24.1-2_armhf.deb /var/cache/apt/archives/libseccomp2_2.5.4-1+rpi1+deb12u1_armhf.deb /var/cache/apt/archives/libsystemd-shared_252.26-1~deb12u2+rpi1_armhf.deb /var/cache/apt/archives/systemd_252.26-1~deb12u2+rpi1_armhf.deb
In the other issue you also said you're seeing chroot: failed to run command ‘/bin/true’: Exec format error
, but I'm not seeing it in the output you posted here.
Can you please post the full output? Is docker running in privileged mode? What is the host OS of the runner?
Yes, the runner is running in privileged mode. The part about '/bin/true'
from the other issue was a mistake on my part. I was running the wrong branch on my local machine and not in Docker, so please ignore that bit.
FROM node:latest AS builder
FROM i386/debian:buster
ENV DEBIAN_FRONTEND noninteractive
# install packages
# NOTE: due to Docker's cache, putting these in 1 line helps to make sure the update runs first when you add or change a package
RUN apt-get -y update && apt-get -y install --no-install-recommends \
git \
vim \
parted \
quilt \
coreutils \
qemu-user-static \
debootstrap \
zerofree \
zip \
dosfstools \
libarchive-tools \
libcap2-bin \
rsync \
grep \
udev \
xz-utils \
curl \
xxd \
file \
kmod \
bc \
binfmt-support \
ca-certificates \
qemu-utils \
kpartx \
build-essential \
ssh \
gpg \
pigz \
arch-test
# remove apt list
RUN rm -rf /var/lib/apt/lists/*
$ uname -a
Linux linux-build-server 5.4.0-1109-azure #115~18.04.1-Ubuntu SMP Mon May 22 20:06:37 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 18.04.6 LTS
Release: 18.04
Codename: bionic
build.sh
OutputThe output is below. However, since cat
in ./build.sh -c config || cat work/Dicon-Raspbian/stage0/debootstrap.log
exits with 0
, the script kept on running after the failure so I don't know how useful the part at the end is.
armhf: not supported on this machine/kernel
WARNING: Only a native build environment is supported. Checking emulated support...
armhf: ok
That's to say - unless you're building natively on an arm machine, your results may vary and support is limited.
However, I may have a rough idea of what's going on. The version of ubuntu you're running there is quite old and I believe it ships with a flaky version of qemu. When the docker guest runs, I believe the host's qemu version is still used as the interpreter for arm binaries. Note that buster's version is likely to be flaky too, maybe in other ways.
I have had success with 22.04 and bookworm.
Thanks, I will try upgrading both the host and the guest.
I updated the host to Ubuntu 22.04 and I updated my docker image base from i386/debian:buster
to i386/debian:bookworm
. Then I got this error:
Module binfmt_misc not loaded in host
Please run:
sudo modprobe binfmt_misc
Then I ran sudo modprobe binfmt_misc
on the host. Then I tried the build again and I just got the same error.
There was one error when building the docker image:
1 warning found (use docker --debug to expand):
- InvalidBaseImagePlatform: Base image i386/debian:bookworm was pulled with platform "linux/386", expected "linux/amd64" for current build (line 3)
but I'm not sure what this means. The i386/debian:bookworm
base image appears to be valid.
Here's the output of lsmod
:
It does include binfmt_misc
.
The check it's failing is grep -q "/proc/sys/fs/binfmt_misc" /proc/mounts
, so it looks like something different is going on in the docker container.
The docker build script in this repo has the following:
if [ ! -f /proc/sys/fs/binfmt_misc/register ]; then
echo "binfmt_misc required but not mounted, trying to mount it..."
if ! mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc ; then
echo "mounting binfmt_misc failed"
exit 1
fi
You can take a look at whatever other workarounds there are - https://github.com/RPi-Distro/pi-gen/blob/master/build-docker.sh
When it start getting into the quirks of docker environments, I'll have to bow out, since it's something I know next to nothing about.
I got this result from the host machine:
$ grep -q "/proc/sys/fs/binfmt_misc" /proc/mounts
$ echo $?
0
$ cat /proc/mounts
sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0
proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0
udev /dev devtmpfs rw,nosuid,relatime,size=8163700k,nr_inodes=2040925,mode=755,inode64 0 0
devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0
tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=1636460k,mode=755,inode64 0 0
/dev/sda1 / ext4 rw,relatime,discard 0 0
securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0
tmpfs /dev/shm tmpfs rw,nosuid,nodev,inode64 0 0
tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k,inode64 0 0
cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0
pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0
bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0
systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=19070 0 0
hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0
mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0
debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0
tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0
fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0
configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0
ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0
/dev/loop0 /snap/core20/2318 squashfs ro,nodev,relatime,errors=continue,threads=single 0 0
/dev/loop1 /snap/lxd/24061 squashfs ro,nodev,relatime,errors=continue,threads=single 0 0
/dev/loop2 /snap/snapd/21759 squashfs ro,nodev,relatime,errors=continue,threads=single 0 0
/dev/sda15 /boot/efi vfat rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=iso8859-1,shortname=mixed,errors=remount-ro 0 0
binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0
/dev/sdb1 /mnt ext4 rw,relatime 0 0
tmpfs /run/snapd/ns tmpfs rw,nosuid,nodev,noexec,relatime,size=1636460k,mode=755,inode64 0 0
nsfs /run/snapd/ns/lxd.mnt nsfs rw 0 0
tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=1636456k,nr_inodes=409114,mode=700,uid=1000,gid=1000,inode64 0 0
$ sudo mount binfmt_misc -t binfmt_misc /proc/sys/fs/binfmt_misc
mount: /proc/sys/fs/binfmt_misc: binfmt_misc already mounted on /proc/sys/fs/binfmt_misc.
grep
finds binfmt_misc
inside /proc/mounts
, and trying to mount it says it's already mounted. So I think this check is fine, unless I'm misunderstanding. I'm doing this on the host machine outside of Docker.
What I'm doing is my build job runs in a Docker container on my build server. The job executes build.sh
. Would it make more or less sense to try using build-docker.sh
in this context? That would be launching a Docker container within a Docker container.
My mistake, I didn't realize check_dependencies
runs within the Docker container.
Anyway, I was able to SSH into my build server, clone pi-gen, and run build-docker.sh
and it got past the steps mentioned above. So I have changed my CI job to use the shell
executor instead of the docker
executor. That means I can run build-docker.sh
from my CI job and it will not be running Docker from within Docker. That build is running now and hopefully it completes successfully...
The build worked. So in summary:
build-docker.sh
script and allow that one to set up the docker container
I am running pi-gen via GitLab CI. Currently I am trying to update from Debian 9 (which was working previously) to Debian 12. However, when I run
build.sh
, I get the following error:Since this is running in Docker via GitLab CI, I don't have a way to check
debootstrap.log
. Is there anything I can do to either a) fix the problem or b) obtain access todebootstrap.log
?