search

abiosoft / colima

Container runtimes on macOS (and Linux) with minimal setup
MIT License
19.77k stars 397 forks source link

colima start fails due to 'QEMU has already exited' - QEMU 8.0.4 #777

Open z0sen opened 1 year ago

z0sen commented 1 year ago

Description

No response

Version

Colima Version: 0.5.5 Lima Version: 0.17.0 Qemu Version: 8.0.4

Operating System

Output of colima status

FATA[0000] colima is not running

Reproduction Steps

  1. Run colima start

Expected behaviour

colima starts without errors.

Additional context

❯ colima start --very-verbose TRAC[0000] cmd ["limactl" "info"] TRAC[0000] cmd ["limactl" "list" "colima" "--json"] INFO[0000] starting colima INFO[0000] runtime: docker TRAC[0000] cmd ["limactl" "list" "colima" "--json"] INFO[0000] preparing network ... context=vm TRAC[0000] cmd ["/usr/local/bin/colima" "daemon" "status" "default"] TRAC[0000] cmd ["/usr/local/bin/colima" "daemon" "stop" "default"] TRAC[0000] cmd ["/usr/local/bin/colima" "daemon" "start" "default" "--gvproxy" "--very-verbose"] TRAC[0000] cmd ["limactl" "list" "colima" "--json"] INFO[0000] starting ... context=vm TRAC[0000] cmd ["limactl" "start" "colima"]

Using the existing instance "colima" [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/xxx/.lima/colima/serial*.log") SSH Local Port: 49426 [hostagent] Waiting for the essential requirement 1 of 5: "ssh" [hostagent] Driver stopped due to error: "exit status 255" [hostagent] Shutting down the host agent [hostagent] failed to exit SSH master [hostagent] Shutting down QEMU with ACPI [hostagent] failed to open the QMP socket "/Users/xxx/.lima/colima/qmp.sock", forcibly killing QEMU [hostagent] QEMU has already exited exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/xxx/.lima/colima/ha.stderr.log") FATA[0001] error starting vm: error at 'starting': exit status 1

AkihiroSuda commented 1 year ago

/Users/xxx/.lima/colima/ha.stderr.log may contain the detail of the issue

z0sen commented 1 year ago

/Users/xxx/.lima/colima/ha.stderr.log below:

{"level":"debug","msg":"Creating iso file /Users/xxx/.lima/colima/cidata.iso","time":"2023-08-13T06:26:57+08:00"}
{"level":"debug","msg":"Using /var/folders/7z/zd9c3g_s7cl8gknyv7jkf0hm0000gn/T/diskfs_iso1842483670 as workspace","time":"2023-08-13T06:26:57+08:00"}
{"level":"debug","msg":"OpenSSH version 9.0.1 detected","time":"2023-08-13T06:26:57+08:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2023-08-13T06:26:57+08:00"}
{"level":"debug","msg":"QEMU version 8.0.4 detected","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"firmware candidates = [/Users/xxx/.local/share/qemu/edk2-x86_64-code.fd /Users/xxx/.colima/_wrapper/4e1b408f843d1c63afbbdcf80c40e4c88d33509f/share/qemu/edk2-x86_64-code.fd /usr/share/OVMF/OVMF_CODE.fd /usr/share/qemu/ovmf-x86_64-code.bin /usr/share/edk2-ovmf/x64/OVMF_CODE.fd]","time":"2023-08-13T06:26:58+08:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/xxx/.lima/colima/serial*.log\")","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"qCmd.Args: [/Users/xxx/.colima/_wrapper/4e1b408f843d1c63afbbdcf80c40e4c88d33509f/bin/qemu-system-x86_64 -m 4096 -cpu host,-pdpe1gb -machine q35,accel=hvf -smp 4,sockets=1,cores=4,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/xxx/.colima/_wrapper/4e1b408f843d1c63afbbdcf80c40e4c88d33509f/share/qemu/edk2-x86_64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/xxx/.lima/colima/basedisk,format=raw,media=cdrom,readonly=on -drive file=/Users/xxx/.lima/colima/diffdisk,if=virtio,discard=on -drive id=cdrom0,if=none,format=raw,readonly=on,file=/Users/xxx/.lima/colima/cidata.iso -device virtio-scsi-pci,id=scsi0 -device scsi-cd,bus=scsi0.0,drive=cdrom0 -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:50200-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:b7:82:b3 -device virtio-rng-pci -display none -device virtio-vga -device virtio-keyboard-pci -device virtio-mouse-pci -device qemu-xhci,id=usb-bus -parallel none -chardev socket,id=char-serial,path=/Users/xxx/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/xxx/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-serial-virtio,path=/Users/xxx/.lima/colima/serialv.sock,server=on,wait=off,logfile=/Users/xxx/.lima/colima/serialv.log -device virtio-serial-pci,id=virtio-serial0,max_ports=1 -device virtconsole,chardev=char-serial-virtio,id=console0 -chardev socket,id=char-qmp,path=/Users/xxx/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/xxx/.lima/colima/qemu.pid]","time":"2023-08-13T06:26:58+08:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/xxx/.lima/_config/user\" -o IdentityFile=\"/Users/xxx/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=xxx -o ControlMaster=auto -o ControlPath=\"/Users/xxx/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 50200 127.0.0.1 -- /bin/bash]","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"qemu[stderr]: qemu-system-x86_64: Unknown Error","time":"2023-08-13T06:26:58+08:00"}
{"level":"info","msg":"Driver stopped due to error: \"exit status 255\"","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 50200: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 50200: Connection refused\\r\\n\": exit status 255","time":"2023-08-13T06:26:58+08:00"}
{"level":"info","msg":"Shutting down the host agent","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"shutting down the SSH master","time":"2023-08-13T06:26:58+08:00"}
{"level":"debug","msg":"executing ssh for exiting the master: /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/xxx/.lima/_config/user\" -o IdentityFile=\"/Users/xxx/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=xxx -o ControlMaster=auto -o ControlPath=\"/Users/xxx/.lima/colima/ssh.sock\" -o ControlPersist=5m -O exit -p 50200 127.0.0.1]","time":"2023-08-13T06:26:58+08:00"}
{"error":"failed to execute `ssh -O exit -p 50200 127.0.0.1`, out=\"Control socket connect(/Users/xxx/.lima/colima/ssh.sock): No such file or directory\\r\\n\": exit status 255","level":"warning","msg":"failed to exit SSH master","time":"2023-08-13T06:26:58+08:00"}
{"level":"info","msg":"Shutting down QEMU with ACPI","time":"2023-08-13T06:26:58+08:00"}
{"error":"dial unix /Users/xxx/.lima/colima/qmp.sock: connect: connection refused","level":"warning","msg":"failed to open the QMP socket \"/Users/xxx/.lima/colima/qmp.sock\", forcibly killing QEMU","time":"2023-08-13T06:26:58+08:00"}
{"level":"info","msg":"QEMU has already exited","time":"2023-08-13T06:26:58+08:00"}
AkihiroSuda commented 1 year ago

qemu-system-x86_64: Unknown Error

Maybe you have to reinstall QEMU. Also please make sure it is not running under Rosetta.

z0sen commented 1 year ago

I've already reinstall QEMU, colima, lima. it's still not woking, and my mac is intel chip, there is no Rosetta

AkihiroSuda commented 1 year ago

Turned out to be a regression in QEMU v8.0.4: https://gitlab.com/qemu-project/qemu/-/issues/1828

Downgrading to v8.0.3 will work:

curl -OSL https://raw.githubusercontent.com/Homebrew/homebrew-core/dc0669eca9479e9eeb495397ba3a7480aaa45c2e/Formula/qemu.rb
brew install ./qemu.rb
z0sen commented 1 year ago

Turned out to be a regression in QEMU v8.0.4: https://gitlab.com/qemu-project/qemu/-/issues/1828

Downgrading to v8.0.3 will work:

curl -OSL https://raw.githubusercontent.com/Homebrew/homebrew-core/dc0669eca9479e9eeb495397ba3a7480aaa45c2e/Formula/qemu.rb
brew install ./qemu.rb

alright, i've fixed it, thanks for ur help

image
rfay commented 1 year ago

@z0sen it's definitely not closed, it will affect all users and upgraders. Please reopen it and improve the title now that it's understood what's going on. Maybe "colima start fails due to 'QEMU has already exited' - QEMU 8.0.4"

Since the homebrew troubles have not been resolved, lots of people will be looking for this issue.

robhung commented 1 year ago

I missed this issue because it was closed, but it doesn't seem to have been resolved yet.. downgrading is a workaround, but certainly not solved

Can confirm that the issue doesn't seem to affect M1 chip, so it's Intel specific - also tried upgrading to lima@0.17.2 which supposedly fixes this problem, ~but still get the same error. Going to keep the downgraded version of qemu for now, and pin it to prevent upgrading~

EDIT: Just needed to follow the instructions of the lima update to sign the binary - I think the solve for this issue should be that colima checks whether the binary is signed, and if it isn't, then prompt the user to sign - as the error being thrown is not clear.

AkihiroSuda commented 1 year ago

The issue is solved:

z0sen commented 1 year ago

@rfay alright i've reopen and rename the title

rfay commented 1 year ago

But now you're too late, you can close it as it was solved. https://github.com/abiosoft/colima/issues/777#issuecomment-1677138032

AkihiroSuda commented 1 year ago

But now you're too late, you can close it as it was solved. #777 (comment)

The regression occurred again:

AkihiroSuda commented 1 year ago

Fixed in the qemu bottle 8.1.0-1 🎉

baggiponte commented 11 months ago

Hi, same problem here. I just got a M3 Pro mac and colima fails to start. However, with my personal M1 mac (same colima qemu and lima versions) it runs smoothly. EDIT: I ran colima delete and colima start and it worked.

AkihiroSuda commented 11 months ago

M3

See

dannylwe commented 11 months ago

This worked for me on M1

SmartManoj commented 5 months ago

Fixed in the qemu bottle 8.1.0-1 🎉

@AkihiroSuda Why is this issue still open if it is fixed?

dantemogrim commented 2 months ago

Just ran into the same issue today after upgrading homebrew & my formulas. As mentioned by some of you I was also helped by running: