qemu-system-x86_64: Unknown Error (Was: `60022: Connection refused SSH`)

[or-israeli]

Hey folks. trying to run limactl start with the default configuration and getting this error

the log file:

{"level":"debug","msg":"firmware candidates = [/usr/local/homebrew/Cellar/qemu/6.2.0/share/qemu/edk2-x86_64-code.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bin]","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"OpenSSH version 8.6.1 detected","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2022-01-11T14:11:09+02:00"} {"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/orisraeli/.lima/default/serial.log\")","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"qCmd.Args: [/usr/local/homebrew/Cellar/qemu/6.2.0/bin/qemu-system-x86_64 -cpu host -machine q35,accel=hvf -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly=on,file=/usr/local/homebrew/Cellar/qemu/6.2.0/share/qemu/edk2-x86_64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/orisraeli/.lima/default/diffdisk,if=virtio -cdrom /Users/orisraeli/.lima/default/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:60022-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:65:16:ff -device virtio-rng-pci -display none -device virtio-vga -device virtio-keyboard-pci -device virtio-mouse-pci -parallel none -chardev socket,id=char-serial,path=/Users/orisraeli/.lima/default/serial.sock,server=on,wait=off,logfile=/Users/orisraeli/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/orisraeli/.lima/default/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-default -pidfile /Users/orisraeli/.lima/default/qemu.pid]","time":"2022-01-11T14:11:09+02:00"} {"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"executing script \"ssh\"","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/orisraeli/.lima/_config/user\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=orisraeli -o ControlMaster=auto -o ControlPath=\"/Users/orisraeli/.lima/default/ssh.sock\" -o ControlPersist=5m -p 60022 127.0.0.1 -- /bin/bash]","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: Unknown Error","time":"2022-01-11T14:11:09+02:00"} {"error":"signal: abort trap","level":"info","msg":"QEMU has exited","time":"2022-01-11T14:11:09+02:00"} {"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 60022: Connection refused\r\n\": exit status 255","time":"2022-01-11T14:11:09+02:00"}

I have the firewall turned off, using the Mac pro max (with the M1 processor). It's seems the config file defined to search for an available port (written with 0 in the SSH PORT field) but it seems my machine is blocking it for some reason

Already updated qemu like specified here if I need to specify any other details please let me know

[0dragosh]

I have the exact same problem.

[AkihiroSuda]

Could you check ~/.lima/default/serial.log?

[vorti2]

Exact same problem, the file ~/.lima/default/serial.log has 0 Bytes on my Apple Silicon with M1Max.

[AkihiroSuda]

{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: Unknown Error","time":"2022-01-11T14:11:09+02:00"}

Maybe your QEMU binary is broken, and need brew uninstall qemu && brew install qemu.

Does qemu-system-x86_64 -accel hvf -m 4096 -cdrom some-linux-iso-image.iso work?

[0dragosh]

qemu-system-x86_64 -accel hvf -m 4096 -cdrom some-linux-iso-image.iso

$ qemu-system-x86_64 -accel hvf -m 4096 -cdrom ubuntu-20.04.3-live-server-amd64.iso
qemu-system-x86_64: -accel hvf: Error: HV_ERROR

[AkihiroSuda]

@0dragosh

If you are using ARM Mac, please test qemu-system-aarch64, not qemu-system-x86_64. Also please make sure your QEMU is v6.2.0 or later.

If you are using Intel Mac, please try reinstalling QEMU. I recommend Homebrew.

[0dragosh]

@0dragosh

If you are using ARM Mac, please test qemu-system-aarch64, not qemu-system-x86_64. Also please make sure your QEMU is v6.2.0 or later.

If you are using Intel Mac, please try reinstalling QEMU. I recommend Homebrew.

I'm on an x64 machine. I have tried reinstalling QEMU.

[demaere-oiie]

I believe I have this problem as well, on an M1 Mac with QEMU 6.2.0.

One thing I've noticed is that there is no ssh.sock in .lima/default; should there be?

Do you have a recommended iso for aarch64?

[demaere-oiie]

I used the exact qemu invocation that I found in ha.stderr.log

/opt/homebrew/bin/qemu-system-aarch64 -cpu host -machine virt,accel=hvf,highmem=off -smp 4,sockets=1,cores=4,threads=1 -m 4096 -drive if=pflash,format=raw,readonly=on,file=/opt/homebrew/share/qemu/edk2-aarch64-code.fd -boot order=c,splash-time=0,menu=on -drive file=/Users/XXX/.lima/default/diffdisk,if=virtio -cdrom /Users/XXX/.lima/default/cidata.iso -netdev user,id=net0,net=[192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:60022-:22](http://192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:60022-:22) -device virtio-net-pci,netdev=net0,mac=XXXXXXX -device virtio-rng-pci -display none -vga none -device ramfb -device qemu-xhci,id=usb-bus -device usb-kbd,bus=usb-bus.0 -device usb-mouse,bus=usb-bus.0 -parallel none -chardev socket,id=char-serial,path=/Users/XXX/.lima/default/serial.sock,server=on,wait=off,logfile=/Users/XXX/.lima/default/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/XXX/.lima/default/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-default -pidfile /Users/XXX/.lima/default/qemu.pid

and looking at the serial.log, it appears there is a problem with the boot process. It fails to boot with UiApp, UEFI Misc Devices, UEFI PXEv[46], UEFI HTTPv[46], and eventually boots into the EFI Internal Shell.

Press ESC in 1 seconds to skip startup.nsh or any other key to continue.
Shell> 

limactl version 0.8.3 QEMU emulator version 6.2.0

[demaere-oiie]

resolved by doing limactl delete default; limactl start — maybe I had had an x86 disk image in default by mistake?

[toshitanian]

I got almost same error which looks to be related to qemu probelm on M1Max Mac. I'm investigating by myself but just for the reference.

Execution log

$ limactl start
INFO[0001] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
INFO[0001] [hostagent] QEMU has exited
FATA[0001] exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "!/.lima/default/ha.stderr.log")

$ cat ~.lima/default/ha.stderr.log
...
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: Unknown Error","time":"2022-02-26T18:54:39+09:00"}
...

serial.log was found but empty.

For the details: https://gist.github.com/toshitanian/d7b71596816d701d8d29047442c3fe8a

Environment

Running on iTerm2 with Rosetta mode.

$ sw_vers
ProductName:    macOS
ProductVersion: 12.2.1
BuildVersion:   21D62

uname -m
x86_64

$ uname -srm
Darwin 21.3.0 x86_64

$  qemu-system-x86_64 --version
QEMU emulator version 6.2.0
Copyright (c) 2003-2021 Fabrice Bellard and the QEMU Project developers

$ brew -v
Homebrew 3.3.16
Homebrew/homebrew-core (git revision 5edf1cd15f7; last commit 2022-02-26)
Homebrew/homebrew-cask (git revision e37ef9b253; last commit 2022-02-26)

$ brew info qemu
qemu: stable 6.2.0 (bottled), HEAD
Emulator for x86 and PowerPC
https://www.qemu.org/
/usr/local/Cellar/qemu/6.2.0 (162 files, 556.7MB) *
  Poured from bottle on 2022-02-25 at 22:45:09
From: https://github.com/Homebrew/homebrew-core/blob/HEAD/Formula/qemu.rb
License: GPL-2.0-only
==> Dependencies
Build: libtool ✔, meson ✘, ninja ✔, pkg-config ✔
Required: glib ✔, gnutls ✔, jpeg ✔, libpng ✔, libslirp ✔, libssh ✔, libusb ✔, lzo ✔, ncurses ✔, nettle ✔, pixman ✔, snappy ✔, vde ✔
==> Options
--HEAD
    Install HEAD version
==> Analytics
install: 42,945 (30 days), 116,086 (90 days), 283,565 (365 days)
install-on-request: 20,489 (30 days), 65,588 (90 days), 203,182 (365 days)
build-error: 48 (30 days)

Running qemu independently

Unknow error for qemu-system-x86_64

$ qemu-system-x86_64 -accel hvf -m 4096 -cdrom ubuntu-20.04.4-live-server-amd64.iso
qemu-system-x86_64: -accel hvf: Unknown Error
[1]    55043 abort      qemu-system-x86_64 -accel hvf -m 4096 -cdrom

$  qemu-system-aarch64 -accel hvf -m 4096 -cdrom /Users/toshiya/Downloads/ubuntu-20.04.4-live-server-amd64.iso
qemu-system-aarch64: No machine specified, and there is no default
Use -machine help to list supported machines

qemu run successfully without -accesl hvf option.

$ qemu-system-x86_64 -m 4096 -cdrom ubuntu-20.04.4-live-server-amd64.iso

[AkihiroSuda]

Rosetta mode.

Unsupported. Please use native binary of QEMU

[toshitanian]

Even though I tried to reinstall lima using Homebrew several times, limactl kept using qemu-system-x86_64. That may be because I was using Homebrew is for x86_64 while I switched to arm on the Terminal. I didn't know I need to install Homebrew for two different environment of x86 and arm.

The solution for me was, install Homebrew arm64, then install qemu and lima using the Homebrew arm64.

• Install Homebrew64
• Install qemu and lima

$ exec arch -arm64e /bin/zsh
$  ~ uname -m
arm64

# Install Homebrew arm64 and switch brew environment
# https://gist.github.com/nrjdalal/e70249bb5d2e9d844cc203fd11f74c55#option-2---manually
$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"
$ echo 'eval "$(/opt/homebrew/bin/brew shellenv)"' >> /Users/$(logname)/.zprofile
$ eval "$(/opt/homebrew/bin/brew shellenv)"

# install qemu and lima using Homebrew arm64
# https://gist.github.com/toricls/d3dd0bec7d4c6ddbcf2d25f211e8cd7b#1-install-qemu--lima
$  brew install qemu
$  brew install lima

Finally!

$ limactl start
$ limactl start --tty=false focal-amd64.yaml
$ limactl shell focal-amd64 uname -a
Linux lima-focal-amd64 5.4.0-99-generic #112-Ubuntu SMP Thu Feb 3 13:50:55 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

[Kit-Ok]

Even though I tried to reinstall lima using Homebrew several times, limactl kept using qemu-system-x86_64. That may be because I was using Homebrew is for x86_64 while I switched to arm on the Terminal. I didn't know I need to install Homebrew for two different environment of x86 and arm.

Thanks a lot for sharing! In my case, which I am using fish terminal, I reinstalled fish from official installer because the old one is installed via X86_64 homebrew, so fish itself is X86_64 too. Installing the homebrew via ARM64 fish terminal worked fine.

[arohner]

I'm hitting this error, and AFAICT I'm using arm64.

$ uname -m
arm64

$ file $(which qemu-system-x86_64)
/opt/homebrew/bin/qemu-system-x86_64: Mach-O 64-bit executable arm64

$ limactl start
? Creating an instance "default" Proceed with the current configuration
INFO[0001] Attempting to download the image from "https://cloud-images.ubuntu.com/releases/22.04/release-20220420/ubuntu-22.04-server-cloudimg-amd64.img"  digest="sha256:de5e632e17b8965f2baf4ea6d2b824788e154d9a65df4fd419ec4019898e15cd"
INFO[0001] Attempting to download the image from "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img"  digest=
600.38 MiB / 600.38 MiB [----------------------------------] 100.00% 16.98 MiB/s
INFO[0037] Downloaded the image from "https://cloud-images.ubuntu.com/releases/22.04/release/ubuntu-22.04-server-cloudimg-amd64.img" 
INFO[0037] Attempting to download the nerdctl archive from "https://github.com/containerd/nerdctl/releases/download/v0.21.0/nerdctl-full-0.21.0-linux-amd64.tar.gz"  digest="sha256:728f9b543374b1b1733f759608e156dbe578d7b140a081084a1f4bfb4f2b3fbf"
217.85 MiB / 217.85 MiB [-----------------------------------] 100.00% 8.82 MiB/s
INFO[0063] Downloaded the nerdctl archive from "https://github.com/containerd/nerdctl/releases/download/v0.21.0/nerdctl-full-0.21.0-linux-amd64.tar.gz" 
ERRO[0064] [hostagent] accelerator "hvf" is not supported by /opt/homebrew/bin/qemu-system-x86_64  fields.level=fatal
FATA[0064] host agent process has exited: exit status 1 

[AkihiroSuda]

@arohner Please make sure you aren’t using Rosetta

[arohner]

Please make sure you aren’t using Rosetta How would I do that?

Doesn't

Mach-O 64-bit executable arm64

prove that I'm using native QEMU?

[AkihiroSuda]

@arohner You might be using x86_64 version of limactl?

[arohner]

$ which limactl
/opt/homebrew/bin/limactl
$ file /opt/homebrew/bin/limactl 
/opt/homebrew/bin/limactl: Mach-O 64-bit executable arm64