search

containers / buildah

A tool that facilitates building OCI images.
https://buildah.io
Apache License 2.0
7.36k stars 777 forks source link

Exec error when trying to target an arm/v7 image with buildah bud #4914

Open nickhobbs94 opened 1 year ago

nickhobbs94 commented 1 year ago

Description

Hi there, I'm new to buildah so please let me know if I'm on the complete wrong track here. I'm trying to build an arm v7 image on an aarch64 machine (actually, in a container with --privilleged that might be relevant).

I can build an arm64 image and an amd64 image. But seem to struggle with the v7 variant. Any hints would be appreciated, thank you :)

Steps to reproduce the issue:

  1. For an example Containerfile here's the contents

    from python
RUN pip install flask

  2. Make a new manifest

    buildah manifest create testmanifest

  3. Build a working image to check everything is working

    buildah bud --tag test:latest --manifest testmanifest --arch arm64 .

  4. Fail to build the v7 variant image (I've tried a few different ways of specifying this, including --platform)

    buildah bud --tag test:latest --manifest testmanifest --arch arm --variant v7 .

Describe the results you received:

exec container process `/bin/sh`: Exec format error
Error: building at STEP "RUN pip install flask": while running runtime: exit status 1

Describe the results you expected: I expected:

Successfully tagged localhost/test:latest

Output of rpm -q buildah or apt list buildah:

buildah/stable,now 1.28.2+ds1-3+b1 arm64 [installed]

Output of buildah version:

Version:         1.28.2
Go Version:      go1.19.8
Image Spec:      1.1.0-rc2
Runtime Spec:    1.0.2-dev
CNI Spec:        1.0.0
libcni Version:  
image Version:   5.23.1
Git Commit:      
Built:           Thu Jan  1 00:00:00 1970
OS/Arch:         linux/arm64
BuildPlatform:   linux/arm64/v8

*Output of `cat /etc/release`:**

PRETTY_NAME="Debian GNU/Linux 12 (bookworm)"
NAME="Debian GNU/Linux"
VERSION_ID="12"
VERSION="12 (bookworm)"
VERSION_CODENAME=bookworm
ID=debian
HOME_URL="https://www.debian.org/"
SUPPORT_URL="https://www.debian.org/support"
BUG_REPORT_URL="https://bugs.debian.org/"

Output of uname -a:

Linux c99ef9653629 6.3.8-200.fc38.aarch64 #1 SMP PREEMPT_DYNAMIC Thu Jun 15 02:15:18 UTC 2023 aarch64 GNU/Linux

Output of cat /etc/containers/storage.conf:

File not found.

flouthoc commented 1 year ago

Your build platform is linux/arm64/v8, which is different from v7. I am not sure if v7 binary can be executed on a v8 build platform. But if it is not allowed I think installing qemu-user-static and building with --platform string should work.

nickhobbs94 commented 1 year ago

@flouthoc thank you. Yeah I am aware they are different. I've tried to build while specifying --platform and it didn't seem to work. What does qemu-user-static do differently?

I guess I'm just confused because I can clearly build different platform architectures with this setup (e.g. amd64) but it doesn't seem to work for different architecture revisions (v8 vs v7).

flouthoc commented 1 year ago

qemu-user-static allows emulation to work for build i.e uses VM when build platform is different from the requested platform.

rhatdan commented 1 year ago

Remove all images used in the build and then start again with the --platform command.

buildah uses images that were previously pulled and if these images are V8 it will use them rather then pulling a new v7 image.

nickhobbs94 commented 1 year ago

Remove all images used in the build and then start again with the --platform command.

Afraid I've already tried this. The problem (I think) is that the system seems to think that an arm v7 image is able to be built on an arm v8 virtual machine. It then hits an exec error because of the mismatch.

Does anybody have an arm v8 platform they'd be willing to try this on? It'd be good if somebody more experienced than myself could reproduce this.

github-actions[bot] commented 1 year ago

A friendly reminder that this issue had no activity for 30 days.

michaels0184 commented 3 months ago

Remove all images used in the build and then start again with the --platform command.

Afraid I've already tried this. The problem (I think) is that the system seems to think that an arm v7 image is able to be built on an arm v8 virtual machine. It then hits an exec error because of the mismatch.

Does anybody have an arm v8 platform they'd be willing to try this on? It'd be good if somebody more experienced than myself could reproduce this.

Did you get it sorted? I too am having same issue building an arm64 Image on a amd64 host, even with QEMU installed.

nickhobbs94 commented 2 months ago

@michaels0184 never did get it sorted unfortunately. Just built my images on different machines.

I don't really get why this is happening. Would love to know how people publish images for all their target platforms normally.

nickhobbs94 commented 2 months ago

@michaels0184 I could build an arm64 image just not the v7 variant. Feel free to chuck your build commands/etc on this ticket, we might be running into different roadblocks.

michaels0184 commented 1 month ago

nickhobbs94 I ended up moving to Azure ACR build tools for building multi architecture images. It comes bundled with the Azure CLI which we already had on our build agents. Buildah required emulation software to be installed at the host level (which we never had access too). I think buildah should just bundle the emulation software into their solution, it would of made my life soo much easier.