qemu-system-aarch64 fails to launch iso on arm64 platform using the same packer .pkr.hcl configuration settings which do boot the iso on qemu-system-x86_64

[HariSekhon]

Overview of the Issue

Packer fails to boot the Ubuntu iso on my M1 Mac running qemu-system-aarch64 which drops into qemu boot shell instead, despite a very similar configuration successfully booting the Ubuntu iso on my Intel Mac running qemu-system-x86_64.

I believe this is more likely to be a packer plugin issue rather than a qemu because if I use UTM which uses qemu under the hood then I can boot the same Ubuntu iso.

Reproduction Steps

git clone https://github.com/HariSekhon/Packer-templates pack

cd pack

packer build ubuntu-arm64.qemu.pkr.hcl

Connect VNC viewer to the random VNC port as output in the log since it doesn't open automatically, as per issue #144, to see it is dropped into a qemu shell and doesn't find / boot the iso.

but if you check the x86_64 version it does boot:

packer build ubuntu-x86_64.qemu.pkr.hcl

Connect VNC viewer to the random port from the output to see it booting.

Plugin and Packer version

Same version running on both my Intel Mac and my M1 Mac

$ packer version
Packer v1.9.1

Simplified Packer Buildfile

x86_64 boots iso:

https://github.com/HariSekhon/Packer-templates/blob/main/ubuntu-x86_64.qemu.pkr.hcl

but arm aarch64 doesn't boot iso:

https://github.com/HariSekhon/Packer-templates/blob/main/ubuntu-arm64.qemu.pkr.hcl

They're close to identical configurations aside from some commented out bits if you diff the two.

Operating system and Environment details

Mac M1 ARM processor running macOS 1.34 Ventura is the one using qemu-system-aarch64 that doesn't boot the iso

$ qemu-system-aarch64 -version
QEMU emulator version 8.0.2

The Intel Mac using qemu-system-x86_64 does boot the iso.

$ qemu-system-x86_64 -version
QEMU emulator version 8.0.2

Log Fragments and crash.log files

Include appropriate log fragments. If the log is longer than a few dozen lines, please include the URL to the gist of the log or use the Github detailed format instead of posting it directly in the issue.

From PACKER_LOG=1 log:

The qemu command run by the failing M1 Mac:

2023/06/12 02:39:02 packer-builder-qemu plugin: Executing /opt/homebrew/bin/qemu-system-aarch64: []string{"-boot", "c", "-m", "3072M", "-vnc", "127.0.0.1:18", "-smp", "3", "-machine", "type=virt,accel=tcg", "-drive", "file=output-ubuntu-22.04-arm64/ubuntu-22.04,if=virtio,cache=writeback,discard=unmap,format=qcow2", "-device", "virtio-net,netdev=user.0", "-name", "ubuntu-22.04", "-netdev", "user,id=user.0,hostfwd=tcp::2408-:22"}

Full Log: https://gist.github.com/HariSekhon/d48fd89d5d1ce1bc957415e174a392f5

The qemu command run by the succeeding Intel Mac:

2023/06/12 02:40:41 packer-builder-qemu plugin: Executing /usr/local/bin/qemu-system-x86_64: []string{"-machine", "type=pc,accel=tcg", "-name", "ubuntu-22.04", "-vnc", "127.0.0.1:41", "-m", "3072M", "-drive", "if=none,file=output-ubuntu-22.04-x86_64/ubuntu-22.04,id=drive0,cache=writeback,discard=unmap,format=qcow2", "-netdev", "user,id=user.0,hostfwd=tcp::2809-:22", "-device", "virtio-scsi-pci,id=scsi0", "-device", "scsi-hd,bus=scsi0.0,drive=drive0", "-device", "virtio-net,netdev=user.0", "-boot", "c", "-smp", "3"}

Full Log: https://gist.github.com/HariSekhon/ffed6bcf30fb48eb63dbb07e96426101

[trungie]

I can confirm this issue on a m1 mac running the same commands.

[signal-09]

I was able to boot by switching disk_interface="virtio", setting accelerator="hvf", and

qemuargs=[           
    ["-cpu", "host"],
    ["-bios", "/opt/homebrew/share/qemu/edk2-aarch64-code.fd"],                                         
    ["-boot", "strict=off"],
    ["-device", "qemu-xhci"],
    ["-device", "usb-kbd"],
    ["-device", "virtio-gpu-pci"],                                                                      
 ]

After boot loader I had to switch view to the serial console, but then I ended in stuck during partitioning due to the disk not visible.

[alisson276]

I have a Apple M1. The only way to put this working is using -monitor stdio, this way the console is not the qemu prompt. Example:

qemu-system-aarch64 -device virtio-net,netdev=user.0 -smp 3 -drive file=ubuntu.qcow2,if=virtio,cache=writeback,discard=unmap,format=qcow2 -m 3000 -M virt,highmem=off -accel hvf -name ubuntu-22.04 -netdev user,id=user.0,hostfwd=tcp::2294-:22 -cdrom ubuntu-22.04.3-live-server-arm64.iso -cpu host -bios QEMU_EFI.fd -boot c -monitor stdio

-cpu host and -accel hvf are required to put this working as well, as stated in https://github.com/hashicorp/packer-plugin-qemu/issues/145#issuecomment-1858523723.