search

suse-edge / edge-image-builder

Tool for creating and configuring a set of images to automate the deployment of Edge environments
Apache License 2.0
33 stars 20 forks source link

Building custom image fails on libguestfs error #148

Closed mpas closed 4 months ago

mpas commented 5 months ago

I am running the builder with the minimal example on a Mac M1 and get an libguestfs error. Not sure what is causing this.

I am building the image using the following configuration:

Please let me know if any other info is needed!

# image definition file
apiVersion: 1.0
image:
  imageType: iso
  arch: x86_64
  baseImage: SLE-Micro.x86_64-5.5.0-Default-SelfInstall-GM.install.iso
  outputImageName: eib-image.iso
# Executed command after successfully building the edge builder docker image
docker run --rm -it \
  -v $IMAGE_DIR:/eib eib:dev /bin/eib \
  -config-file $CONFIG_FILE \
  -config-dir /eib \
  -build-dir /eib/_build
# console output
Building ISO Image with parameters:
IMAGE_DIR = /Users/mpas/Development/scratchpad/eib-test
CONFIG_FILE = eib-config-iso.yaml
BUILD_DIR = /Users/mpas/Development/scratchpad/eib-test/_build
Generating image customization components...
Identifier ................... [SUCCESS]
Custom Files ................. [SKIPPED]
Time ......................... [SKIPPED]
Network ...................... [SKIPPED]
Users ........................ [SKIPPED]
Proxy ........................ [SKIPPED]
Rpm .......................... [SKIPPED]
Systemd ...................... [SKIPPED]
Elemental .................... [SKIPPED]
Suma ......................... [SKIPPED]
Embedded Artifact Registry ... [SKIPPED]
Keymap ....................... [SUCCESS]
Kubernetes ................... [SKIPPED]
Certificates ................. [SKIPPED]
Building ISO image...
Kernel Params ................ [SKIPPED]
Error building ISO image, check the logs under the build directory for more information.
# raw-build.log
libguestfs: error: appliance closed the connection unexpectedly.
This usually means the libguestfs appliance crashed.
Do:
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
libguestfs: error: /usr/bin/qemu-system-aarch64 exited with error status 1.
To see full error messages you may need to enable debugging.
Do:
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
libguestfs: error: guestfs_launch failed.
This usually means the libguestfs appliance failed to start or crashed.
Do:
  export LIBGUESTFS_DEBUG=1 LIBGUESTFS_TRACE=1
and run the command again.  For further information, read:
  http://libguestfs.org/guestfs-faq.1.html#debugging-libguestfs
You can also run 'libguestfs-test-tool' and post the *complete* output
into a bug report or message to the libguestfs mailing list.
e-minguez commented 5 months ago

I think it maybe related to the fact eib is not supported on arm64 (yet), see https://github.com/suse-edge/edge-image-builder/issues/68

Just in case, can you run the same command with debug enabled?

Note: maybe we need to check the architecture and fail gracefully instead of letting libvirt fail 😅 /cc @jdob

mpas commented 5 months ago

Will run with debug enabled..

What I actually want to achieve is to run eib on my Arm machine and create a x64 ISO image. That is something we have been doing with previous SLE Micro versions / setups that worked nicely.

In that setup we added the following docker arguments to produce compatible ISO images.

    PLATFORM_ARG="--platform linux/amd64"
    BUILD_CMD="docker buildx build"

So if there are any tips on how to achieve this, then that would be great!

mpas commented 5 months ago

Have run with debug enabled like in the linked issue.

# eib-build.log
2024-02-05T14:19:05.346Z    INFO    combustion/network.go:50    Configuring network component...
2024-02-05T14:19:05.346Z    INFO    combustion/network.go:54    Skipping network component, configuration is not provided
2024-02-05T14:19:05.346Z    INFO    combustion/rpm.go:32    Skipping RPM component. Configuration is not provided
2024-02-05T14:19:05.346Z    INFO    combustion/elemental.go:29  Skipping elemental registration component, configuration is not provided
2024-02-05T14:19:05.347Z    INFO    combustion/certificates.go:28   skipping certificate configuration, no certificates provided
2024-02-05T14:19:24.390Z    FATAL   eib/main.go:96  An error occurred building the image: modifying the raw image inside of the ISO: running the image modification script: exit status 1
main.main
    /src/cmd/eib/main.go:96
runtime.main
    /usr/lib64/go/1.21/src/runtime/proc.go:267
# raw-build.log
libguestfs: trace: set_verbose true
libguestfs: trace: set_verbose = 0
libguestfs: create: flags = 0, handle = 0xaaab14666f90, program = guestfish
libguestfs: trace: add_drive "/eib/_build/build-Feb06_08-19-46/raw-extract/SLE-Micro.raw" "format:raw"
libguestfs: trace: add_drive = 0
libguestfs: trace: is_config
libguestfs: trace: is_config = 1
libguestfs: trace: launch
libguestfs: trace: max_disks
libguestfs: trace: max_disks = 255
libguestfs: trace: get_tmpdir
libguestfs: trace: get_tmpdir = "/tmp"
libguestfs: trace: version
libguestfs: trace: version = <struct guestfs_version = major: 1, minor: 48, release: 6, extra: , >
libguestfs: trace: get_backend
libguestfs: trace: get_backend = "direct"
libguestfs: launch: program=guestfish
libguestfs: launch: version=1.48.6
libguestfs: launch: backend registered: libvirt
libguestfs: launch: backend registered: direct
libguestfs: launch: backend=direct
libguestfs: launch: tmpdir=/tmp/libguestfsbRgghY
libguestfs: launch: umask=0022
libguestfs: launch: euid=0
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: begin building supermin appliance
libguestfs: run supermin
libguestfs: command: run: /usr/bin/supermin
libguestfs: command: run: \ --build
libguestfs: command: run: \ --verbose
libguestfs: command: run: \ --if-newer
libguestfs: command: run: \ --lock /var/tmp/.guestfs-0/lock
libguestfs: command: run: \ --copy-kernel
libguestfs: command: run: \ -f ext2
libguestfs: command: run: \ --host-cpu aarch64
libguestfs: command: run: \ /usr/lib64/guestfs/supermin.d
libguestfs: command: run: \ -o /var/tmp/.guestfs-0/appliance.d
supermin: version: 5.2.2
supermin: rpm: detected RPM version 4.14
supermin: rpm: detected RPM architecture aarch64
supermin: rpm: detected zypper version 1.14.66
supermin: package handler: opensuse/rpm
supermin: acquiring lock on /var/tmp/.guestfs-0/lock
supermin: build: /usr/lib64/guestfs/supermin.d
supermin: reading the supermin appliance
supermin: build: visiting /usr/lib64/guestfs/supermin.d/base.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/daemon.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/excludefiles type uncompressed excludefiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/hostfiles type uncompressed hostfiles
supermin: build: visiting /usr/lib64/guestfs/supermin.d/init.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/packages type uncompressed packages
supermin: build: visiting /usr/lib64/guestfs/supermin.d/udev-rules.tar.gz type gzip base image (tar)
supermin: build: visiting /usr/lib64/guestfs/supermin.d/zz-scripts.tar.gz type gzip base image (tar)
supermin: mapping package names to installed packages
supermin: resolving full list of package dependencies
supermin: build: 133 packages, including dependencies
supermin: build: 6092 files
supermin: build: 4306 files, after matching excludefiles
supermin: build: 4307 files, after adding hostfiles
supermin: build: 4298 files, after removing unreadable files
supermin: build: 4305 files, after munging
supermin: kernel: looking for kernel using environment variables ...
supermin: kernel: looking for kernels in /lib/modules/*/vmlinuz ...
supermin: kernel: looking for kernels in /boot ...
supermin: kernel: kernel version of /boot/vmlinux-5.14.21-150500.55.44-default.gz = 5.14.21-150500.55.44-default (from filename)
supermin: kernel: picked modules path /lib/modules/5.14.21-150500.55.44-default
supermin: kernel: kernel version of /boot/Image-5.14.21-150500.55.44-default = 5.14.21-150500.55.44-default (from filename)
supermin: kernel: picked modules path /lib/modules/5.14.21-150500.55.44-default
supermin: kernel: picked vmlinuz /boot/vmlinux-5.14.21-150500.55.44-default.gz
supermin: kernel: kernel_version 5.14.21-150500.55.44-default
supermin: kernel: modpath /lib/modules/5.14.21-150500.55.44-default
supermin: ext2: creating empty ext2 filesystem '/var/tmp/.guestfs-0/appliance.d.taysixy4/root'
supermin: ext2: populating from base image
supermin: ext2: copying files from host filesystem
supermin: ext2: copying kernel modules
supermin: ext2: creating minimal initrd '/var/tmp/.guestfs-0/appliance.d.taysixy4/initrd'
supermin: ext2: wrote 44 modules to minimal initrd
supermin: renaming /var/tmp/.guestfs-0/appliance.d.taysixy4 to /var/tmp/.guestfs-0/appliance.d
libguestfs: finished building supermin appliance
libguestfs: begin testing qemu features
libguestfs: trace: get_cachedir
libguestfs: trace: get_cachedir = "/var/tmp"
libguestfs: checking for previously cached test results of /usr/bin/qemu-system-aarch64, in /var/tmp/.guestfs-0
libguestfs: command: run: /usr/bin/qemu-system-aarch64
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -help
libguestfs: command: run: /usr/bin/qemu-system-aarch64
libguestfs: command: run: \ -display none
libguestfs: command: run: \ -machine virt,accel=kvm:tcg
libguestfs: command: run: \ -device ?
libguestfs: command: run: echo '{ "execute": "qmp_capabilities" }' '{ "execute": "query-qmp-schema" }' '{ "execute": "quit" }'  | QEMU_AUDIO_DRV=none "/usr/bin/qemu-system-aarch64" -display none -machine "virt,accel=kvm:tcg" -qmp stdio
libguestfs: did not understand QMP monitor output from /usr/bin/qemu-system-aarch64 (ignored)
libguestfs: command: run: echo '{ "execute": "qmp_capabilities" }' '{ "execute": "query-kvm" }' '{ "execute": "quit" }'  | QEMU_AUDIO_DRV=none "/usr/bin/qemu-system-aarch64" -display none -machine "virt,accel=kvm:tcg" -qmp stdio
libguestfs: did not understand QMP monitor output from /usr/bin/qemu-system-aarch64 (ignored)
libguestfs: saving test results
libguestfs: QMP parse error: '[' or '{' expected near end of file (ignored)
libguestfs: QMP parse error: '[' or '{' expected near end of file (ignored)
libguestfs: qemu version: 7.1
libguestfs: qemu mandatory locking: yes
libguestfs: qemu KVM: enabled
libguestfs: trace: get_backend_setting "force_tcg"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_backend_setting "force_kvm"
libguestfs: trace: get_backend_setting = NULL (error)
libguestfs: trace: get_sockdir
libguestfs: trace: get_sockdir = "/tmp"
libguestfs: finished testing qemu features
libguestfs: trace: get_backend_setting "gdb"
libguestfs: trace: get_backend_setting = NULL (error)
/usr/bin/qemu-system-aarch64 \
    -global virtio-blk-pci.scsi=off \
    -no-user-config \
    -nodefaults \
    -display none \
    -machine virt,gic-version=host,accel=kvm:tcg \
    -cpu host \
    -m 1280 \
    -no-reboot \
    -rtc driftfix=slew \
    -kernel /var/tmp/.guestfs-0/appliance.d/kernel \
    -initrd /var/tmp/.guestfs-0/appliance.d/initrd \
    -object rng-random,filename=/dev/urandom,id=rng0 \
    -device virtio-rng-pci,rng=rng0 \
    -device virtio-scsi-pci,id=scsi \
    -drive file=/eib/_build/build-Feb06_08-19-46/raw-extract/SLE-Micro.raw,cache=writeback,format=raw,id=hd0,if=none \
    -device scsi-hd,drive=hd0 \
    -drive file=/var/tmp/.guestfs-0/appliance.d/root,snapshot=on,id=appliance,cache=unsafe,if=none \
    -device scsi-hd,drive=appliance \
    -device virtio-serial-pci \
    -serial stdio \
    -chardev socket,path=/tmp/libguestfsw1Hf2W/guestfsd.sock,id=channel0 \
    -device virtserialport,chardev=channel0,name=org.libguestfs.channel.0 \
    -append "panic=1 console=ttyAMA0 earlyprintk=pl011,,0x9000000 ignore_loglevel efi-rtc=noprobe edd=off udevtimeout=6000 udev.event-timeout=6000 no_timer_check printk.time=1 cgroup_disable=memory usbcore.nousb cryptomgr.notests tsc=reliable 8250.nr_uarts=1 root=UUID=3f1e326b-a98a-4bbd-9c8a-4ac276bd2297 selinux=0 guestfs_verbose=1 TERM=xterm"
Could not access KVM kernel module: No such file or directory
qemu-system-aarch64: failed to initialize kvm: No such file or directory
qemu-system-aarch64: falling back to tcg
qemu-system-aarch64: gic-version=host requires KVM
libguestfs: error: appliance closed the connection unexpectedly, see earlier error messages
libguestfs: child_cleanup: 0xaaab14666f90: child process died
libguestfs: sending SIGTERM to process 177
libguestfs: error: /usr/bin/qemu-system-aarch64 exited with error status 1, see debug messages above
libguestfs: error: guestfs_launch failed, see earlier error messages
libguestfs: trace: launch = -1 (error)
libguestfs: trace: close
libguestfs: closing guestfs handle 0xaaab14666f90 (state 0)
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsbRgghY
libguestfs: command: run: rm
libguestfs: command: run: \ -rf /tmp/libguestfsw1Hf2W
mpas commented 5 months ago

Related to the message that Could not access KVM kernel module: No such file or directory. I tried setting the option LIBGUESTFS_BACKEND_SETTINGS=force_tcg inside the modify-raw-image.sh.tpl. But this gives another error..

I found this option reading the thread below. https://www.mail-archive.com/debian-bugs-dist@lists.debian.org/msg1740696.html

atanasdinov commented 4 months ago

Thanks for the interest in this. Unfortunately, the initial release will not support aarch64 but we're definitely looking into introducing it at a later point.

jdob commented 4 months ago

Closing as a duplicate of the feature request for aarch #193