solo-io / packer-plugin-arm-image

Packer plugin for ARM images
Apache License 2.0
425 stars 104 forks source link

How to bind mount the chroot directory onto the chroot / #149

Open jseparovic opened 2 years ago

jseparovic commented 2 years ago

Hi,

Is there a way to bind mount the chroot directory onto / ?

I need to get docker pull working in the chroot, and in order for docker pull to work properly I need to bind mount the chroot directory onto the chroot /

Ie. mount -o bind chroot-ubuntu chroot-ubuntu/

Here's the full working chroot example:

mkdir chroot-ubuntu
debootstrap --variant=buildd jammy chroot-ubuntu
mount -o bind chroot-ubuntu chroot-ubuntu/
mount --rbind /sys chroot-ubuntu/sys
mount --rbind /dev chroot-ubuntu/dev
mount -t proc /proc chroot-ubuntu/proc
chroot chroot-ubuntu /bin/bash
apt update
apt install -y iptables curl
curl -fsSL https://get.docker.com | sh
/usr/bin/dockerd --iptables=False &
sleep 2
docker pull alpine

Here is the thread where the chroot bind mount was suggested to fix the docker pull issue. https://github.com/moby/moby/issues/34817

jseparovic commented 2 years ago

I added the chroot bind mount with:

{
    "builders": [
        {
            "type": "arm-image",
            "iso_url": "/images/2022-09-06-raspios-bullseye-arm64-lite.img",
            "iso_checksum": "sha256:13ece30029a7725807a4760a21a2150c379b9ea963a72937ef7de5ce35211a7f",
            "target_image_size": 5368709120,
            "qemu_binary": "qemu-aarch64-static",
            "mount_path": "/tmp/pi_image",
            "chroot_mounts": [
                ["bind", "/tmp/pi_image", "/"],
                ["proc", "proc", "/proc"],
                ["sysfs", "sysfs", "/sys"],
                ["bind", "/dev", "/dev"],
                ["devpts", "devpts", "/dev/pts"],
                ["binfmt_misc", "binfmt_misc", "/proc/sys/fs/binfmt_misc"]
            ]
        }
    ],
    "provisioners": [
        {
            "type": "shell",
            "inline": [
                "apt update",
                "apt install -y iptables curl cgroupfs-mount",
                "curl -fsSL https://get.docker.com | sh",
                "cgroupfs-mount",
                "nohup /usr/bin/dockerd --iptables=False &",
                "sleep 3",
                "docker pull alpine"
            ]
        }
    ]
}

But I still see the error at the end of the docker pull process

    arm-image: Using default tag: latest
    arm-image: latest: Pulling from library/alpine
    arm-image: 9b18e9b68314: Pulling fs layer
    arm-image: 9b18e9b68314: Verifying Checksum
    arm-image: 9b18e9b68314: Download complete
==> arm-image: time="2022-09-20T02:02:11.523089549+01:00" level=info msg="Attempting next endpoint for pull after error: failed to register layer: Error processing tar file(exit status 1): "
==> arm-image: failed to register layer: Error processing tar file(exit status 1):
jseparovic commented 2 years ago

If figured out a workaround for this. Instead of starting docker in the chroot env, build as normal, and then load the image as a second disk on an Ubuntu VM. Install docker on the VM and change the base dir to the rasp pi image docker dir. Then restart docker and do the docker pull. It will setup the docker image on the rasp pi image. Do a sync and unmount before killing the VM.