search

abiosoft / colima

Container runtimes on macOS (and Linux) with minimal setup
MIT License
19.77k stars 397 forks source link

Error starting with --network-address #458

Open liaden opened 2 years ago

liaden commented 2 years ago

Description

Attempting to start --network-address fails. Additional starts without the flag fail with a different error message. Removing ~/.colima allows for a successful start, without the network address.

Version

Colima Version: 0.4.6 Lima Version: 0.12.0, 0.13.0 Qemu Version: 7.1.0

Operating System

Reproduction Steps

  1. colima start --network-address

Expected behaviour

Colima starts correctly.

Additional context

First start:

INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-27T09:57:15-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0015] creating and starting ...                     context=vm
WARN[0015] error setting up routable IP address: vmnet ptp socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.ptp: no such file or directory
> Using cache "/Users/joel.johnson/Library/Caches/lima/download/by-url-sha256/e9bac04e9bdb31be4d3de1506d97eb60d59d9ad1a2d97f2b21f760e06f3e4408/data"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 57347
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'creating and starting': exit status 1

ha.stderr.log:

{"level":"debug","msg":"Creating iso file /Users/joel.johnson/.lima/colima/cidata.iso","time":"2022-10-27T09:57:15-04:00"}
{"level":"debug","msg":"Using /var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/diskfs_iso413631167 as workspace","time":"2022-10-27T09:57:15-04:00"}
{"level":"debug","msg":"firmware candidates = [/Users/joel.johnson/.local/share/qemu/edk2-aarch64-code.fd /Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd /usr/share/AAVMF/AAVMF_CODE.fd /usr/share/qemu-efi-aarch64/QEMU_EFI.fd]","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"OpenSSH version 8.6.1 detected","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2022-10-27T09:57:17-04:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/joel.johnson/.lima/colima/serial.log\")","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"qCmd.Args: [/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-aarch64 -m 2048 -cpu host -machine virt,accel=hvf,highmem=off -smp 2,sockets=1,cores=2,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/joel.johnson/.lima/colima/basedisk,media=cdrom,readonly=on -drive file=/Users/joel.johnson/.lima/colima/diffdisk,if=virtio,discard=on -cdrom /Users/joel.johnson/.lima/colima/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:57347-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:56:07:0a -device virtio-rng-pci -display none -vga none -device ramfb -device qemu-xhci,id=usb-bus -device usb-kbd,bus=usb-bus.0 -device usb-mouse,bus=usb-bus.0 -parallel none -chardev socket,id=char-serial,path=/Users/joel.johnson/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/joel.johnson/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/joel.johnson/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/joel.johnson/.lima/colima/qemu.pid]","time":"2022-10-27T09:57:17-04:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"qemu[stderr]: time=\"2022-10-27T09:57:17-04:00\" level=fatal msg=\"dial unix /Users/joel.johnson/.colima/default/daemon/gvproxy.sock: connect: no such file or directory\"","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2022-10-27T09:57:17-04:00"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/joel.johnson/.lima/_config/user\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_ed25519\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=lima -o ControlMaster=auto -o ControlPath=\"/Users/joel.johnson/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 57347 127.0.0.1 -- /bin/bash]","time":"2022-10-27T09:57:17-04:00"}
{"level":"debug","msg":"stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 57347: Connection refused\\r\\n\", err=failed to execute script \"ssh\": stdout=\"\", stderr=\"ssh: connect to host 127.0.0.1 port 57347: Connection refused\\r\\n\": exit status 255","time":"2022-10-27T09:57:17-04:00"}

Second start:

INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-27T10:01:35-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0015] error setting up routable IP address: vmnet ptp socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.ptp: no such file or directory
INFO[0015] starting ...                                  context=vm
> Using the existing instance "colima"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 57370
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'starting': exit status 1

which is obviously due to the colima.yaml being updated with the --network-address flag.

abiosoft commented 2 years ago

Are you on macOS Ventura as well? If yes, we can close this in favour of https://github.com/abiosoft/colima/issues/457

abiosoft commented 2 years ago

Can you kindly try with the current development version?

brew install --HEAD colima
liaden commented 2 years ago

@abiosoft I am not on Ventura. It still errors after upgrading. Results are below.

console logging:

TRAC[0000] cmd ["limactl" "info"]
TRAC[0000] cmd ["limactl" "list" "colima" "--json"]
INFO[0000] starting colima
INFO[0000] runtime: docker
TRAC[0000] cmd ["limactl" "list" "colima" "--json"]
INFO[0000] preparing network ...                         context=vm
TRAC[0000] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0000] cmd ["/opt/homebrew/bin/colima" "daemon" "start" "default" "--vmnet" "--gvproxy" "--verbose"]
TRAC[0000] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0003] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0006] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0009] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0012] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0015] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-10-30T12:42:10-04:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0015] error setting up routable IP address: vmnet socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.sock: no such file or directory
INFO[0015] starting ...                                  context=vm
TRAC[0015] cmd ["limactl" "start" "colima"]
> Using the existing instance "colima"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 51585
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] QEMU has exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0017] error starting vm: error at 'starting': exit status 1

~/.lima/colima/ha.stderr.log

{"level":"debug","msg":"Creating iso file /Users/joel.johnson/.lima/colima/cidata.iso","time":"2022-10-30T12:42:10-04:00"}
{"level":"debug","msg":"Using /var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/diskfs_iso3203651481 as workspace","time":"2022-10-30T12:42:10-04:00"}
{"level":"debug","msg":"firmware candidates = [/Users/joel.johnson/.local/share/qemu/edk2-aarch64-code.fd /Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd /usr/share/AAVMF/AAVMF_CODE.fd /usr/share/qemu-efi-aarch64/QEMU_EFI.fd]","time":"2022-10-30T12:42:11-04:00"}
{"level":"debug","msg":"OpenSSH version 8.6.1 detected","time":"2022-10-30T12:42:11-04:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2022-10-30T12:42:11-04:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/joel.johnson/.lima/colima/serial.log\")","time":"2022-10-30T12:42:11-04:00"}
{"level":"debug","msg":"qCmd.Args: [/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-aarch64 -m 2048 -cpu host -machine virt,accel=hvf,highmem=off -smp 2,sockets=1,cores=2,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/joel.johnson/.lima/colima/basedisk,media=cdrom,readonly=on -drive file=/Users/joel.johnson/.lima/colima/diffdisk,if=virtio,discard=on -cdrom /Users/joel.johnson/.lima/colima/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:51585-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:56:07:0a -device virtio-rng-pci -display none -vga none -device ramfb -device qemu-xhci,id=usb-bus -device usb-kbd,bus=usb-bus.0 -device usb-mouse,bus=usb-bus.0 -parallel none -chardev socket,id=char-serial,path=/Users/joel.johnson/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/joel.johnson/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/joel.johnson/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/joel.johnson/.lima/colima/qemu.pid]","time":"2022-10-30T12:42:11-04:00"}
{"level":"debug","msg":"qemu[stderr]: time=\"2022-10-30T12:42:12-04:00\" level=fatal msg=\"error connecting to gvproxy socket: dial unix /Users/joel.johnson/.colima/default/daemon/gvproxy.sock: connect: no such file or directory\"","time":"2022-10-30T12:42:12-04:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2022-10-30T12:42:12-04:00"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2022-10-30T12:42:12-04:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2022-10-30T12:42:12-04:00"}
{"level":"debug","msg":"executing ssh for script \"ssh\": /usr/bin/ssh [ssh -F /dev/null -o IdentityFile=\"/Users/joel.johnson/.lima/_config/user\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_ed25519\" -o IdentityFile=\"/Users/joel.johnson/.ssh/id_rsa\" -o StrictHostKeyChecking=no -o UserKnownHostsFile=/dev/null -o NoHostAuthenticationForLocalhost=yes -o GSSAPIAuthentication=no -o PreferredAuthentications=publickey -o Compression=no -o BatchMode=yes -o IdentitiesOnly=yes -o Ciphers=\"^aes128-gcm@openssh.com,aes256-gcm@openssh.com\" -o User=lima -o ControlMaster=auto -o ControlPath=\"/Users/joel.johnson/.lima/colima/ssh.sock\" -o ControlPersist=5m -p 51585 127.0.0.1 -- /bin/bash]","time":"2022-10-30T12:42:12-04:00"}
abiosoft commented 2 years ago

This is very strange.

Can you start again with network address to reproduce the error and share the contents of the daemon directory?

ls -lah ~/.colima/default/daemon
liaden commented 2 years ago

ls -lah ~/.colima/default/daemon

total 4.0K
drwxr-xr-x 3 joel.johnson 387623102   96 Oct 30 12:41 .
drwxr-xr-x 4 joel.johnson 387623102  128 Oct 30 12:39 ..
-rw-r--r-- 1 joel.johnson 387623102 1.6K Oct 30 12:41 daemon.log

cat ~/.colima/default/daemon/daemon.log

time="2022-10-30T12:39:30-04:00" level=info msg="- - - - - - - - - - - - - - -"
time="2022-10-30T12:39:30-04:00" level=info msg="daemon started by colima"
time="2022-10-30T12:39:30-04:00" level=info msg="Run `pkill -F /Users/joel.johnson/.colima/default/daemon/daemon.pid` to kill the daemon"
time="2022-10-30T12:39:30-04:00" level=info msg="waiting for clients..."
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required
time="2022-10-30T12:39:30-04:00" level=error msg="error starting vmnet: error running vmnet: exit status 1"
time="2022-10-30T12:39:30-04:00" level=info msg="terminate signal received"
time="2022-10-30T12:39:30-04:00" level=fatal msg="context canceled"
time="2022-10-30T12:41:55-04:00" level=info msg="- - - - - - - - - - - - - - -"
time="2022-10-30T12:41:55-04:00" level=info msg="daemon started by colima"
time="2022-10-30T12:41:55-04:00" level=info msg="Run `pkill -F /Users/joel.johnson/.colima/default/daemon/daemon.pid` to kill the daemon"
time="2022-10-30T12:41:55-04:00" level=info msg="waiting for clients..."
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required
time="2022-10-30T12:41:55-04:00" level=error msg="error starting vmnet: error running vmnet: exit status 1"
time="2022-10-30T12:41:55-04:00" level=info msg="terminate signal received"
time="2022-10-30T12:41:55-04:00" level=fatal msg="context canceled"
abiosoft commented 2 years ago

Does it prompt you for a password for setting up network? Can you deleted the permission file and the extracted network files, then try again?

sudo rm -r /etc/sudoers.d/colima /opt/colima
liaden commented 2 years ago

It did not prompt me for sudo. It has before, but even after the suggested command, it is not prompting.

abiosoft commented 2 years ago

Oops, I made a typo in the command. I've fixed it now.

Rajendraladkat1919 commented 2 years ago

Hey can you do one thing.

  1. Just uninstall the colima and docker cli.
  2. remove all .docker , .limb, .colima from the host machine.
  3. install the latest colima and docker-cli usging brew.
  4. colima start --network-address
  5. Above command will prompt you for the password.
liaden commented 2 years ago

Reran with the above steps plus removing /etc/sudoers.d/colima and /opt/colima. It failed same as before, but it did prompt for sudo.

STDOUT:

TRAC[0000] cmd ["limactl" "info"]
TRAC[0000] cmd ["limactl" "list" "colima" "--json"]
TRAC[0000] error retrieving running instance: error retrieving instance: unexpected end of JSON input
INFO[0000] starting colima
INFO[0000] runtime: docker
INFO[0000] preparing network ...                         context=vm
INFO[0000] dependencies missing for setting up reachable IP address  context=vm
INFO[0000] sudo password may be required                 context=vm
TRAC[0000] cmd int ["sudo" "mkdir" "-p" "/etc/sudoers.d"]
Password:
TRAC[0008] cmd int ["sudo" "sh" "-c" "cat > /etc/sudoers.d/colima"]
TRAC[0008] cmd int ["sudo" "mkdir" "-p" "/opt/colima"]
TRAC[0008] cmd int ["sudo" "sh" "-c" "cd /opt/colima && tar xfz /var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/vmnet.tar.gz2890914088 2>/dev/null"]
TRAC[0008] cmd int ["sudo" "mkdir" "-p" "/opt/colima/run"]
TRAC[0008] cmd ["ln" "-sf" "/opt/homebrew/bin/colima" "/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-aarch64"]
TRAC[0008] cmd ["ln" "-sf" "/opt/homebrew/bin/colima" "/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-x86_64"]
TRAC[0008] cmd ["ln" "-sf" "/opt/homebrew/share" "/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share"]
TRAC[0008] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0008] cmd ["/opt/homebrew/bin/colima" "daemon" "start" "default" "--vmnet" "--gvproxy" "--verbose"]
TRAC[0008] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0011] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0014] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0017] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0020] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
TRAC[0023] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "default"]
WARN[0023] error setting up network dependencies: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-11-10T13:49:37-05:00\" level=fatal msg=\"pid file not found: stat /Users/joel.johnson/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0023] creating and starting ...                     context=vm
WARN[0023] error setting up routable IP address: vmnet socket file not found: stat /Users/joel.johnson/.colima/default/daemon/vmnet.sock: no such file or directory
TRAC[0023] cmd ["limactl" "start" "--tty=false" "/var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/colima.yaml"]
> Terminal is not available, proceeding without opening an editor
> "Attempting to download the image from \"https://github.com/abiosoft/alpine-lima/releases/download/colima-v0.4.5-1/alpine-lima-clm-3.16.2-aarch64.iso\"" digest="sha512:a79cdb3de4b297ddbc944f9f5de18ea34697b122facc94c7048324267929c19c4502fee8d2495f11f77867e2629dc721feed487affc96025212b0cccd49b28d1"
> Using cache "/Users/joel.johnson/Library/Caches/lima/download/by-url-sha256/e9bac04e9bdb31be4d3de1506d97eb60d59d9ad1a2d97f2b21f760e06f3e4408/data"
> [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/joel.johnson/.lima/colima/serial.log")
> SSH Local Port: 64794
> [hostagent] QEMU has exited
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/joel.johnson/.lima/colima/ha.stderr.log")
FATA[0026] error starting vm: error at 'creating and starting': exit status 1

~/.lima/colima/ha.stderr.log

{"level":"debug","msg":"executing [ssh-keygen -t ed25519 -q -N  -f /Users/joel.johnson/.lima/_config/user]","time":"2022-11-10T13:49:38-05:00"}
{"level":"debug","msg":"Creating iso file /Users/joel.johnson/.lima/colima/cidata.iso","time":"2022-11-10T13:49:38-05:00"}
{"level":"debug","msg":"Using /var/folders/1n/fvnyc01554s0240g36s382jh6nvfqs/T/diskfs_iso2155425194 as workspace","time":"2022-11-10T13:49:38-05:00"}
{"level":"debug","msg":"firmware candidates = [/Users/joel.johnson/.local/share/qemu/edk2-aarch64-code.fd /Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd /usr/share/AAVMF/AAVMF_CODE.fd /usr/share/qemu-efi-aarch64/QEMU_EFI.fd]","time":"2022-11-10T13:49:39-05:00"}
{"level":"debug","msg":"OpenSSH version 8.6.1 detected","time":"2022-11-10T13:49:39-05:00"}
{"level":"debug","msg":"AES accelerator seems available, prioritizing aes128-gcm@openssh.com and aes256-gcm@openssh.com","time":"2022-11-10T13:49:39-05:00"}
{"level":"info","msg":"Starting QEMU (hint: to watch the boot progress, see \"/Users/joel.johnson/.lima/colima/serial.log\")","time":"2022-11-10T13:49:39-05:00"}
{"level":"debug","msg":"qCmd.Args: [/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/bin/qemu-system-aarch64 -m 2048 -cpu host -machine virt,accel=hvf,highmem=off -smp 2,sockets=1,cores=2,threads=1 -drive if=pflash,format=raw,readonly=on,file=/Users/joel.johnson/.colima/_wrapper/3a9197e1ca3cd2da076da2b473d7a7eb118e2cca/share/qemu/edk2-aarch64-code.fd -boot order=d,splash-time=0,menu=on -drive file=/Users/joel.johnson/.lima/colima/basedisk,media=cdrom,readonly=on -drive file=/Users/joel.johnson/.lima/colima/diffdisk,if=virtio,discard=on -cdrom /Users/joel.johnson/.lima/colima/cidata.iso -netdev user,id=net0,net=192.168.5.0/24,dhcpstart=192.168.5.15,hostfwd=tcp:127.0.0.1:64794-:22 -device virtio-net-pci,netdev=net0,mac=52:55:55:56:07:0a -device virtio-rng-pci -display none -vga none -device ramfb -device qemu-xhci,id=usb-bus -device usb-kbd,bus=usb-bus.0 -device usb-mouse,bus=usb-bus.0 -parallel none -chardev socket,id=char-serial,path=/Users/joel.johnson/.lima/colima/serial.sock,server=on,wait=off,logfile=/Users/joel.johnson/.lima/colima/serial.log -serial chardev:char-serial -chardev socket,id=char-qmp,path=/Users/joel.johnson/.lima/colima/qmp.sock,server=on,wait=off -qmp chardev:char-qmp -name lima-colima -pidfile /Users/joel.johnson/.lima/colima/qemu.pid]","time":"2022-11-10T13:49:39-05:00"}
{"level":"debug","msg":"qemu[stderr]: time=\"2022-11-10T13:49:39-05:00\" level=fatal msg=\"error connecting to gvproxy socket: dial unix /Users/joel.johnson/.colima/default/daemon/gvproxy.sock: connect: no such file or directory\"","time":"2022-11-10T13:49:39-05:00"}
{"error":"exit status 1","level":"info","msg":"QEMU has exited","time":"2022-11-10T13:49:39-05:00"}
{"level":"info","msg":"Waiting for the essential requirement 1 of 5: \"ssh\"","time":"2022-11-10T13:49:39-05:00"}
{"level":"debug","msg":"executing script \"ssh\"","time":"2022-11-10T13:49:39-05:00"}

~/.colima/default/daemon/daemon.log

time="2022-11-10T13:49:22-05:00" level=info msg="- - - - - - - - - - - - - - -"
time="2022-11-10T13:49:22-05:00" level=info msg="daemon started by colima"
time="2022-11-10T13:49:22-05:00" level=info msg="Run `pkill -F /Users/joel.johnson/.colima/default/daemon/daemon.pid` to kill the daemon"
time="2022-11-10T13:49:22-05:00" level=info msg="Using search domains: [omada.me omadahealth.net]"
time="2022-11-10T13:49:22-05:00" level=info msg="waiting for clients..."
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required
time="2022-11-10T13:49:22-05:00" level=error msg="error starting vmnet: error running vmnet: exit status 1"
time="2022-11-10T13:49:22-05:00" level=info msg="terminate signal received"
time="2022-11-10T13:49:22-05:00" level=fatal msg="context canceled"
codrutpopescu commented 2 years ago

I have the same issue on Ventura M1. The colima version with --HEAD option colima does start but there is no IP address assigned.

codrutpopescu commented 2 years ago

Here is my daemon.log if it helps:

time="2022-11-11T21:40:10+01:00" level=info msg="- - - - - - - - - - - - - - -"
time="2022-11-11T21:40:10+01:00" level=info msg="daemon started by colima"
time="2022-11-11T21:40:10+01:00" level=info msg="Run `pkill -F /Users/codrut/.colima/oraxe/daemon/daemon.pid` to kill the daemon"
time="2022-11-11T21:40:10+01:00" level=info msg="waiting for clients..."
dyld[41864]: Library not loaded: /opt/colima/lib/libvdeplug.3.dylib
  Referenced from: <E0AE3F70-E540-36C8-9DA8-6846A194A2E7> /opt/colima/bin/vde_vmnet
  Reason: tried: '/opt/colima/lib/libvdeplug.3.dylib' (mach-o file, but is an incompatible architecture (have 'arm64e', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/opt/colima/lib/libvdeplug.3.dylib' (no such file), '/opt/colima/lib/libvdeplug.3.dylib' (mach-o file, but is an incompatible architecture (have 'arm64e', need 'arm64')), '/usr/local/lib/libvdeplug.3.dylib' (no such file), '/usr/lib/libvdeplug.3.dylib' (no such file, not in dyld cache)
time="2022-11-11T21:40:10+01:00" level=error msg="error starting vmnet: error running vmnet: signal: abort trap"
time="2022-11-11T21:40:10+01:00" level=info msg="terminate signal received"
time="2022-11-11T21:40:10+01:00" level=fatal msg="context canceled"
rfay commented 2 years ago

Reason: tried: '/opt/colima/lib/libvdeplug.3.dylib' (mach-o file, but is an incompatible architecture (have 'arm64e', need 'arm64')), '/System/Volumes/Preboot/Cryptexes/OS/opt/colima/lib/libvdeplug.3.dylib' (no such file), '/opt/colima/lib/libvdeplug.3.dylib' (mach-o file, but is an incompatible architecture (have 'arm64e', need 'arm64')), '/usr/local/lib/libvdeplug.3.dylib' (no such file), '/usr/lib/libvdeplug.3.dylib' (no such file, not in dyld cache)

You just can't run that image, as it has an incompatible architecture, whatever arm64e is.

Really, oracle mysql image isn't going to work for you on an arm64 (mac M1) machine.

codrutpopescu commented 2 years ago

Sorry, the above error is without installing using --HEAD. I am not using mysql. This is simply colima. Here are the steps:

brew uninstall colima docker docker-compose qemu
brew autoremove
brew cleanup
sudo rm -rf /opt/colima
rm -rf .docker .lima .colima

brew install --HEAD colima docker docker-compose
colima start --arch x86_64 --memory 4 --network-address oraxe

colima list
PROFILE    STATUS     ARCH      CPUS    MEMORY    DISK     RUNTIME    ADDRESS
oraxe      Running    x86_64    2       4GiB      60GiB    docker

as you can see there is no IP address assigned

cat .colima/oraxe/daemon/daemon.log
time="2022-11-11T22:27:34+01:00" level=info msg="- - - - - - - - - - - - - - -"
time="2022-11-11T22:27:34+01:00" level=info msg="daemon started by colima"
time="2022-11-11T22:27:34+01:00" level=info msg="Run `pkill -F /Users/codrut/.colima/oraxe/daemon/daemon.pid` to kill the daemon"
time="2022-11-11T22:27:34+01:00" level=info msg="waiting for clients..."
on_accept(): vmnet_return_t VMNET_INVALID_ARGUMENT
vmnet_write: Undefined error: 0
time="2022-11-11T22:27:37+01:00" level=info msg="new connection from  to /Users/codrut/.colima/oraxe/daemon/gvproxy.sock"
rfay commented 2 years ago

Sorry, I did read it wrong, usually this is seen with oracle. But the problem is the same. You seem to be trying to use an architecture arm64e, please change that in your config file to arm64. AFAIK arm64e not even a valid architecture. (I guess I saw "oraexe" and thought I read oracle).

To recreate your situation, rename the .colima folder to .colima.bak and the lima folder to .lima.bak and then go through your process.

abiosoft commented 2 years ago

@altitudedashboard the error you shared should be gone with --HEAD.

Can you try starting with --network-driver slirp and see if it's same behaviour (using the --HEAD version)?

codrutpopescu commented 2 years ago

@abiosoft This works! Thank you! Steps followed: clean everything:

brew uninstall qemu lima colima docker docker-compose
brew autoremove
brew cleanup
sudo rm -rf /opt/colima
rm -rf /opt/homebrew/etc/gnutls
rm -rf /opt/homebrew/etc/unbound
rm -rf .lima .colima .docker

starts from scratch:

brew install qemu lima docker docker-compose
mkdir -p ~/.docker/cli-plugins
ln -sfn /opt/homebrew/opt/docker-compose/bin/docker-compose ~/.docker/cli-plugins/docker-compose
brew install --HEAD colima
colima start --arch x86_64 --memory 4 --network-driver slirp --network-address oraxe
colima list                                                                         
PROFILE    STATUS     ARCH      CPUS    MEMORY    DISK     RUNTIME    ADDRESS
oraxe      Running    x86_64    2       4GiB      60GiB    docker     192.168.106.2
codrutpopescu commented 2 years ago

The above seems to work intermittently, as today trying to start colima doesn't revel an IP address

sparhomenko commented 2 years ago

Have the same problem on the latest (13.0.1) macOS and M1.

$ colima stop
$ colima delete -f
$ brew uninstall colima
$ brew autoremove
$ sudo rm -rf .lima .colima /etc/sudoers.d/colima /opt/colima
$ brew install --HEAD colima
$ colima start --network-address
$ colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME    ADDRESS
default    Running    aarch64    2       2GiB      60GiB    docker
$ cat ~/.colima/default/daemon/daemon.log
...
on_accept(): vmnet_return_t VMNET_INVALID_ARGUMENT
vmnet_write: Undefined error: 0

Having the same error with all combinations of runtimes and network drivers (--runtime containerd --network-driver slirp, --runtime docker --network-driver slirp, --runtime containerd --network-driver gvproxy, --runtime docker --network-driver gvproxy).

codrutpopescu commented 2 years ago

just try also with the latest colima version:

brew install --HEAD colima

and also perform a reinstall of qemu after colima as it might fix some bugs

brew reinstall qemu

abiosoft commented 2 years ago

@sparhomenko kindly use the --HEAD version. macOS Ventura has broken network address assignment in v0.4.6.

sparhomenko commented 2 years ago

Apologies, I should have mentioned that I was already using HEAD with the latest commit:

$ brew upgrade colima
Warning: colima HEAD-412d01b already installed

Reinstalling qemu also doesn't help (I was actually already doing that as brew autoremove removed qemu and brew install --HEAD colima reinstalled it). Perhaps there are any additional debug logs I can enable to provide more information?

melnyk-viktor commented 1 year ago

Having a similar problem... Clearing and reinstalling/updating (with dependencies) didn't help.

godfrey-philip commented 1 year ago

I am also having the same problem, my logs are exactly the same however wiping the directories and reinstalling from HEAD works "intermittently" as described above but to the negative. Meaning: if I blow it all away and start over sometimes it errors out completely and sometimes it starts with no routable IP, I am not able to get a valid IP at all.

jhult commented 1 year ago

It seems like there may be an issue with the default profile.

If I run the following (profile = main), I can get --network-address to work.

brew remove colima
brew install --HEAD colima
colima delete default
colima start --runtime containerd --network-address main
colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME       ADDRESS
main       Running    aarch64    2       2GiB      60GiB    containerd    192.168.106.4

If I don't specify a profile name and start Colima (colima start --runtime containerd --network-address), then it throws this error:

INFO[0000] starting colima
INFO[0000] runtime: containerd
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2022-12-29T17:29:16-06:00\" level=fatal msg=\"pid file not found: stat /Users/jonathan/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0015] creating and starting ...                     context=vm
WARN[0015] error setting up reachable IP address: vmnet socket file not found: stat /Users/jonathan/.colima/default/daemon/vmnet.sock: no such file or directory
PARENTELF commented 1 year ago

Hello folks,

Version: Colima Version: HEAD-fe625fb Lima Version: limactl version 0.14.2 Qemu Version: 7.2.0

Operating System macOS Apple M2 (ventura 13.1)

I would like to have some advices to solve the issue about the option --network-address, I have tried all the possibilities, a clean installation and all the stuff described above.

but I'm still getting this information :

  cat .colima/default/daemon/daemon.log

  time="2022-11-11T22:27:34+01:00" level=info msg="- - - - - - - - - - - - - - -"
  time="2022-11-11T22:27:34+01:00" level=info msg="daemon started by colima"
  time="2022-11-11T22:27:34+01:00" level=info msg="Run `pkill -F /Users/XXX/.colima/default/daemon/daemon.pid` to kill the daemon"
  time="2022-11-11T22:27:34+01:00" level=info msg="waiting for clients..."
  on_accept(): vmnet_return_t VMNET_INVALID_ARGUMENT

  vmnet_write: Undefined error: 0

and no attribution of IP is made.

Thanks in advance.

abiosoft commented 1 year ago

@PARENTELF I am struggling with properly reproducing this as well.

It seems to happen (for me) in one of the following scenarios.

The main issue is that the behaviour is not consistent, it works most time but when it breaks I am usually unable to resolve it until a macOS restart.

Since you are on macOS 13 Ventura, you can give the VZ driver a try --vm-type vz. While VZ is more experimental, the network address is more stable.

colima delete # required for switching vm types
colima start --vm-type vz --network-address

PS: kindly update to latest head as it includes a fix to a regression introduced in HEAD-fe625fb

PARENTELF commented 1 year ago

Hello thanks for your quick reply,

Colima updated: colima version HEAD-ed49fef

when I add this extra option :--vm-type vz, I fall into the following case:

note: after the installation of colima I have give a restart. 

hostagent] [VZ] - vm state change: running
> [hostagent] 2023/01/03 14:54:33 tcpproxy: for incoming conn 127.0.0.1:49378, error dialing "192.168.5.15:22": connect tcp 192.168.5.15:22: no route
> [hostagent] Waiting for the essential requirement 1 of 3: "ssh"
> [hostagent] 2023/01/03 14:54:43 tcpproxy: for incoming conn 127.0.0.1:49379, error dialing "192.168.5.15:22": connect tcp 192.168.5.15:22: connection was refused

wich is also related #533

thanks!

abiosoft commented 1 year ago

when I add this extra option :--vm-type vz, I fall into the following case:

note: after the installation of colima I have give a restart.

hostagent] [VZ] - vm state change: running
> [hostagent] 2023/01/03 14:54:33 tcpproxy: for incoming conn 127.0.0.1:49378, error dialing "192.168.5.15:22": connect tcp 192.168.5.15:22: no route
> [hostagent] Waiting for the essential requirement 1 of 3: "ssh"
> [hostagent] 2023/01/03 14:54:43 tcpproxy: for incoming conn 127.0.0.1:49379, error dialing "192.168.5.15:22": connect tcp 192.168.5.15:22: connection was refused

@PARENTELF this is not "an issue" except it got stuck. It's due to the VM booting and ssh server not yet running, you can disregard it.

PARENTELF commented 1 year ago

ok thanks,

the problem is persisting: 

➜  ~ colima ls
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME    ADDRESS
default    Running    x86_64     4       8GiB      60GiB    docker     
main       Running    aarch64    2       2GiB      60GiB    docker     
oraxe      Running    aarch64    2       2GiB      60GiB    docker     
oraxe1     Running    aarch64    2       2GiB      60GiB    docker     
oraxe2     Running    aarch64    2       2GiB      60GiB    docker

thank you

PARENTELF commented 1 year ago

Hello, I would like to give some extra information:

after a fresh installation I always see :

 colima start --cpu 4 --memory 8 --arch x86_64 --vm-type qemu --network-address
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0000] preparing network ...                         context=vm
INFO[0000] dependencies missing for setting up reachable IP address  context=vm
INFO[0000] sudo password may be required                 context=vm
Password:
INFO[0009] creating and starting ...                     context=vm

what are the missing dependencies? :
INFO[0000] dependencies missing for setting up reachable IP address context=vm

In advance thank you for you response

best regards

msankhala commented 1 year ago

I am still getting this error when I enable the network address true in the default config file. I have reinstalled everything from scratch by following the step given in https://github.com/abiosoft/colima/issues/458#issuecomment-1312378827 

❯ colima start
INFO[0000] starting colima
INFO[0000] runtime: containerd
INFO[0000] preparing network ...                         context=vm
WARN[0015] error starting network: error at 'preparing network': error running [/usr/local/bin/colima daemon status default], output: "time=\"2023-01-07T12:08:54+05:30\" level=fatal msg=\"pid file not found: stat /Users/xxx/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0015] error setting up reachable IP address: vmnet socket file not found: stat /Users/xxx/.colima/default/daemon/vmnet.sock: no such file or directory
INFO[0015] starting ...                                  context=vm
> Using the existing instance "colima"
> errors inspecting instance: [host agent is running but driver is not]
FATA[0015] error starting vm: error at 'starting': exit status 1

My ~/.colima/default/colima.yaml file.

# Number of CPUs to be allocated to the virtual machine.
# Default: 2
cpu: 2

# Size of the disk in GiB to be allocated to the virtual machine.
# NOTE: changing this has no effect after the virtual machine has been created.
# Default: 60
disk: 60

# Size of the memory in GiB to be allocated to the virtual machine.
# Default: 2
memory: 2

# Architecture of the virtual machine (x86_64, aarch64, host).
# Default: host
arch: host

# Container runtime to be used (docker, containerd).
# Default: docker
runtime: containerd

# Kubernetes configuration for the virtual machine.
kubernetes:
  # Enable kubernetes.
  # Default: false
  enabled: false

  # Kubernetes version to use.
  # This needs to exactly match a k3s version https://github.com/k3s-io/k3s/releases
  # Default: latest stable release
  version: v1.24.3+k3s1

  # Disable k3s features [coredns servicelb traefik local-storage metrics-server].
  # All features are enabled unless disabled.
  #
  # EXAMPLE - disable traefik and metrics-server
  # disable: [traefik, metrics-server]
  #
  # Default: [traefik]
  disable:
    - traefik

# Auto-activate on the Host for client access.
# Setting to true does the following on startup
#  - sets as active Docker context (for Docker runtime).
#  - sets as active Kubernetes context (if Kubernetes is enabled).
# Default: true
autoActivate: true

# Network configurations for the virtual machine.
network:
  # Assign reachable IP address to the virtual machine.
  # NOTE: this is currently macOS only and ignored on Linux.
  # Default: false
  address: true

  # Custom DNS resolvers for the virtual machine.
  #
  # EXAMPLE
  # dns: [8.8.8.8, 1.1.1.1]
  #
  # Default: []
  dns: []

  # DNS hostnames to resolve to custom targets using the internal resolver.
  # This setting has no effect if a custom DNS resolver list is supplied above.
  # It does not configure the /etc/hosts files of any machine or container.
  # The value can be an IP address or another host.
  #
  # EXAMPLE
  # dnsHosts:
  #   example.com: 1.2.3.4
  dnsHosts:
    host.docker.internal: host.lima.internal

  # Network driver to use (slirp, gvproxy), (requires vmType `qemu`)
  #   - slirp is the default user mode networking provided by Qemu
  #   - gvproxy is an alternative to VPNKit based on gVisor https://github.com/containers/gvisor-tap-vsock
  # Default: gvproxy
  driver: gvproxy

# ===================================================================== #
# ADVANCED CONFIGURATION
# ===================================================================== #

# Forward the host's SSH agent to the virtual machine.
# Default: false
forwardAgent: true

# Docker daemon configuration that maps directly to daemon.json.
# https://docs.docker.com/engine/reference/commandline/dockerd/#daemon-configuration-file.
# NOTE: some settings may affect Colima's ability to start docker. e.g. `hosts`.
#
# EXAMPLE - disable buildkit
# docker:
#   features:
#     buildkit: false
#
# EXAMPLE - add insecure registries
# docker:
#   insecure-registries:
#     - myregistry.com:5000
#     - host.docker.internal:5000
#
# Colima default behaviour: buildkit enabled
# Default: {}
docker: {}

# Virtual Machine type (qemu, vz)
# NOTE: this is macOS 13 only. For Linux and macOS <13.0, qemu is always used.
#
# vz is macOS virtualization framework and requires macOS 13
#
# Default: qemu
vmType: qemu

# Utilise rosetta for amd64 emulation (requires m1 mac and vmType `vz`)
# Default: false
rosetta: false

# Volume mount driver for the virtual machine (virtiofs, 9p, sshfs).
#
# virtiofs is limited to macOS and vmType `vz`. It is the fastest of the options.
#
# 9p is the recommended and the most stable option for vmType `qemu`.
#
# sshfs is faster than 9p but the least reliable of the options (when there are lots
# of concurrent reads or writes).
#
# Default: virtiofs (for vz), sshfs (for qemu)
mountType: sshfs

# The CPU type for the virtual machine (requires vmType `qemu`).
# Options available for host emulation can be checked with: `qemu-system-$(arch) -cpu help`.
# Instructions are also supported by appending to the cpu type e.g. "qemu64,+ssse3".
# Default: host
cpuType: host

# For a more general purpose virtual machine, Ubuntu container is optionally provided
# as a layer on the virtual machine.
# The underlying virtual machine is still accessible via `colima ssh --layer=false` or running `colima` in
# the Ubuntu session.
#
# Default: false
layer: false

# Custom provision scripts for the virtual machine.
# Provisioning scripts are executed on startup and therefore needs to be idempotent.
#
# EXAMPLE - script exected as root
# provision:
#   - mode: system
#     script: apk add htop vim
#
# EXAMPLE - script exected as user
# provision:
#   - mode: user
#     script: |
#       [ -f ~/.provision ] && exit 0;
#       echo provisioning as $USER...
#       touch ~/.provision
#
# Default: []
provision: []

# Modify ~/.ssh/config automatically to include a SSH config for the virtual machine.
# SSH config will still be generated in ~/.colima/ssh_config regardless.
# Default: true
sshConfig: true

# Configure volume mounts for the virtual machine.
# Colima mounts user's home directory by default to provide a familiar
# user experience.
#
# EXAMPLE
# mounts:
#   - location: ~/secrets
#     writable: false
#   - location: ~/projects
#     writable: true
#
# Colima default behaviour: $HOME and /tmp/colima are mounted as writable.
# Default: []
mounts: []

# Environment variables for the virtual machine.
#
# EXAMPLE
# env:
#   KEY: value
#   ANOTHER_KEY: another value
#
# Default: {}
env: {}
jhult commented 1 year ago

@msankhala, if you try using a profile other than default (see here), does that help?

pecigonzalo commented 1 year ago

@PARENTELF it comes from here https://github.com/abiosoft/colima/blob/main/environment/vm/lima/lima.go#L119 which calls then here https://github.com/abiosoft/colima/blob/1c55a0979c8d958578dfd9691957fb3958135c80/daemon/process/gvproxy/gvproxy.go#L208-L213.

@msankhala try removing /opt/colima as suggested here https://github.com/abiosoft/colima/issues/590

xdubx commented 1 year ago

Hi same problem. on a m1 mac os v.12 after clean install like codrutP mentioned. I get an ip adress on the vm. After stop and start i get the error the docker pid. is not in the directory. I only at the first run get a password promt.

` user@SIT-SMBP-C2FR2D ~ % colima start oraxe --very-verbose TRAC[0000] cmd ["limactl" "info"]
TRAC[0000] cmd ["limactl" "list" "colima-oraxe" "--json"] INFO[0000] starting colima [profile=oraxe]
INFO[0000] runtime: docker
TRAC[0000] cmd ["limactl" "list" "colima-oraxe" "--json"] INFO[0000] preparing network ... context=vm INFO[0000] dependencies missing for setting up reachable IP address context=vm INFO[0000] sudo password may be required context=vm TRAC[0000] cmd int ["sudo" "mkdir" "-p" "/etc/sudoers.d"] Password: TRAC[0010] cmd int ["sudo" "sh" "-c" "cat > /etc/sudoers.d/colima"] TRAC[0010] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] TRAC[0010] cmd ["/opt/homebrew/bin/colima" "daemon" "start" "oraxe" "--vmnet" "--verbose"] TRAC[0010] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] TRAC[0013] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] TRAC[0017] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] TRAC[0020] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] TRAC[0023] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] TRAC[0026] cmd ["/opt/homebrew/bin/colima" "daemon" "status" "oraxe"] WARN[0026] error setting up network dependencies: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status oraxe], output: "time=\"2023-02-02T11:29:58+01:00\" level=fatal msg=\"pid file not found: stat /Users/user/.colima/oraxe/daemon/daemon.pid: no such file or directory\"", err: "exit status 1" context=vm WARN[0026] error setting up reachable IP address: vmnet socket file not found: stat /Users/user/.colima/oraxe/daemon/vmnet.sock: no such file or directory INFO[0026] starting ... context=vm TRAC[0026] cmd ["limactl" "start" "colima-oraxe"]

Using the existing instance "colima-oraxe" [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/user/.lima/colima-oraxe/serial.log") SSH Local Port: 57052 [hostagent] Waiting for the essential requirement 1 of 5: "ssh" [hostagent] Waiting for the essential requirement 1 of 5: "ssh" [hostagent] The essential requirement 1 of 5 is satisfied [hostagent] Waiting for the essential requirement 2 of 5: "user session is ready for ssh" [hostagent] The essential requirement 2 of 5 is satisfied [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed" [hostagent] The essential requirement 3 of 5 is satisfied [hostagent] Waiting for the essential requirement 4 of 5: "/etc/fuse.conf (/etc/fuse3.conf) to contain \"user_allow_other\"" [hostagent] The essential requirement 4 of 5 is satisfied [hostagent] Waiting for the essential requirement 5 of 5: "the guest agent to be running" [hostagent] The essential requirement 5 of 5 is satisfied [hostagent] Mounting "/Users/user” on "/Users/user” [hostagent] Mounting "/tmp/colima-oraxe" on "/tmp/colima-oraxe" [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished" [hostagent] Forwarding "/var/run/docker.sock" (guest) to "/Users/user/.colima/oraxe/docker.sock" (host) [hostagent] Forwarding "/run/lima-guestagent.sock" (guest) to "/Users/user/.lima/colima-oraxe/ga.sock" (host) [hostagent] The final requirement 1 of 1 is satisfied READY. Run limactl shell colima-oraxe to open the shell. TRAC[0068] cmd ["lima" "sudo" "cat" "/etc/hosts"]
TRAC[0068] cmd ["lima" "sudo" "cat" "/etc/hosts"]
INFO[0068] provisioning ... context=docker TRAC[0068] cmd ["lima" "sudo" "mkdir" "-p" "/etc/docker"] TRAC[0068] cmd int ["lima" "sudo" "sh" "-c" "cat > /etc/docker/daemon.json"] TRAC[0068] cmd ["docker" "context" "inspect" "colima-oraxe"] TRAC[0068] cmd ["docker" "context" "create" "colima-oraxe" "--description" "colima [profile=oraxe]" "--docker" "host=unix:///Users/user/.colima/oraxe/docker.sock"] colima-oraxe Successfully created context "colima-oraxe" TRAC[0068] cmd ["docker" "context" "use" "colima-oraxe"] colima-oraxe Current context is now "colima-oraxe" Warning: DOCKER_HOST environment variable overrides the active context. To use "colima-oraxe", either set the global --context flag, or unset DOCKER_HOST environment variable. INFO[0068] starting ... context=docker TRAC[0068] cmd ["lima" "sudo" "service" "docker" "start"]

  • /var/log/docker.log: creating file
  • /var/log/docker.log: correcting owner
  • Starting Docker Daemon ... [ ok ] TRAC[0068] cmd ["lima" "sudo" "docker" "info"]
    TRAC[0074] cmd ["lima" "sudo" "docker" "info"]
    TRAC[0074] cmd ["lima" "sudo" "cat" "/etc/colima/colima.json"] TRAC[0074] cmd ["lima" "sudo" "mkdir" "-p" "/etc/colima"] TRAC[0074] cmd ["lima" "sudo" "mkdir" "-p" "/etc/colima"] TRAC[0074] cmd int ["lima" "sudo" "sh" "-c" "cat > /etc/colima/colima.json"] TRAC[0074] cmd ["lima" "sudo" "cat" "/etc/colima/colima.json"] TRAC[0074] cmd ["lima" "sudo" "mkdir" "-p" "/etc/colima"] TRAC[0074] cmd ["lima" "sudo" "mkdir" "-p" "/etc/colima"] TRAC[0074] cmd int ["lima" "sudo" "sh" "-c" "cat > /etc/colima/colima.json"] INFO[0074] done
    TRAC[0074] cmd ["limactl" "list" "--json"]
    TRAC[0074] cmd ["limactl" "show-ssh" "--format" "config" "colima-oraxe"] TRAC[0074] cmd ["limactl" "list" "colima-oraxe" "--json"] `

After restart. ` PROFILE STATUS ARCH CPUS MEMORY DISK RUNTIME ADDRESS

oraxe Running x86_64 2 2GiB 60GiB docker
`

deamon.log time="2023-02-02T11:20:16+01:00" level=info msg="- - - - - - - - - - - - - - -" time="2023-02-02T11:20:16+01:00" level=info msg="daemon started by colima" time="2023-02-02T11:20:16+01:00" level=info msg="Run/usr/bin/pkill -F /Users/user/.colima/oraxe/daemon/daemon.pidto kill the daemon" pidfile_open: Resource temporarily unavailable time="2023-02-02T11:20:16+01:00" level=error msg="error starting vmnet: error running vmnet: exit status 1" time="2023-02-02T11:20:16+01:00" level=info msg="terminate signal received" time="2023-02-02T11:20:16+01:00" level=fatal msg="context canceled" time="2023-02-02T11:23:09+01:00" level=info msg="- - - - - - - - - - - - - - -"

guillemurano commented 1 year ago

> colima delete default

This doesn't work for me either 

% colima delete default
are you sure you want to delete colima and all settings? [y/N] y
INFO[0002] deleting colima                              
INFO[0002] done                   
% colima start --runtime containerd --network-address main
INFO[0000] starting colima [profile=main]               
INFO[0000] runtime: containerd                          
INFO[0000] preparing network ...                         context=vm
INFO[0003] creating and starting ...                     context=vm
INFO[0055] provisioning ...                              context=containerd
INFO[0055] starting ...                                  context=containerd
INFO[0060] done   
% colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME       ADDRESS
main       Running    aarch64    2       2GiB      60GiB    containerd

I also tried % colima start --network-address --network-driver slirp without success

ryanplusplus commented 1 year ago

We were having this issue and discovered that having a multipass VM running causes this behavior. I assume that other VM hosts could cause a similar conflict.

kewilson commented 1 year ago

same thing happens on intel mac .. posted in 449 as well.

This is after I removed & reinstalled everything per the suggestions in this and 449, think there was another around vmnet.

colima version
colima version HEAD-fff2ddd
git commit: fff2ddd76045fd31432a3a69321bf1f13e7fad1d

runtime: docker
arch: x86_64
client: v23.0.1
server: v20.10.20

regular network driver

colima start --arch x86_64 --network-address
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0000] preparing network ...                         context=vm
INFO[0000] dependencies missing for setting up reachable IP address  context=vm
INFO[0000] sudo password may be required                 context=vm
Password:
WARN[0027] error setting up network dependencies: error at 'preparing network': error running [/usr/local/bin/colima daemon status default], output: "time=\"2023-03-21T14:12:48-05:00\" level=fatal msg=\"pid file not found: stat /Users/SNIPPED/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0027] creating and starting ...                     context=vm
WARN[0027] error setting up reachable IP address: vmnet socket file not found: stat /Users/SNIPPED/.colima/default/daemon/vmnet.sock: no such file or directory 
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
> [hostagent] Waiting for the essential requirement 2 of 5: "user session is ready for ssh"
> did not receive an event with the "running" status
FATA[0628] error starting vm: error at 'creating and starting': exit status 1

slirp driver

colima start --arch x86_64 --network-address --network-driver slirp
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0000] preparing network ...                         context=vm
INFO[0000] dependencies missing for setting up reachable IP address  context=vm
INFO[0000] sudo password may be required                 context=vm
Password:
WARN[0025] error setting up network dependencies: error at 'preparing network': error running [/usr/local/bin/colima daemon status default], output: "time=\"2023-03-21T14:33:04-05:00\" level=fatal msg=\"pid file not found: stat /Users/SNIPPED/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
INFO[0025] creating and starting ...                     context=vm
WARN[0025] error setting up reachable IP address: vmnet socket file not found: stat /Users/SNIPPED/.colima/default/daemon/vmnet.sock: no such file or directory 
INFO[0046] provisioning ...                              context=docker
INFO[0046] starting ...                                  context=docker
INFO[0052] done                                         

colima status
INFO[0000] colima is running using QEMU                 
INFO[0000] arch: x86_64                                 
INFO[0000] runtime: docker                              
INFO[0000] mountType: sshfs                             
INFO[0000] socket: unix:///Users/SNIPPED/.colima/default/docker.sock 

colima list
PROFILE    STATUS     ARCH      CPUS    MEMORY    DISK     RUNTIME    ADDRESS
default    Running    x86_64    2       2GiB      60GiB    docker
abiosoft commented 1 year ago

Can anyone try the latest development version brew install --head colima and see if the issue still persists?

Thanks.

DezzardHD commented 1 year ago

Hi @abiosoft , I followed the instructions of codRutP. Same error message as before.

Specs: Apple MacBook Pro M1 Ventura 13.0.1 (22A400) ❯ docker --version Docker version 23.0.2, build 569dd73db1 ❯ colima --version colima version HEAD-65ee3d2

act28 commented 1 year ago

Same issue when running multiple instance with -p flag on MacBook Pro M1 Ventura 13.2.1 using stock default.yaml

tcpproxy: for incoming conn 127.0.0.1:64218, error dialing "192.168.5.15:22": connect tcp 192.168.5.15: no route

Tried using --network-address, same issue.

jandresen99 commented 1 year ago

Hi! I have the same error when using --network-address

colima start --network-address
INFO[0000] starting colima                              
INFO[0000] runtime: docker                              
INFO[0000] preparing network ...                         context=vm
WARN[0016] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status default], output: "time=\"2023-04-10T23:27:19-03:00\" level=fatal msg=\"pid file not found: stat /Users/jandresen/.colima/default/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0016] error setting up reachable IP address: vmnet socket file not found: stat /Users/jandresen/.colima/default/daemon/vmnet.sock: no such file or directory 
INFO[0016] starting ...                                  context=vm
INFO[0039] provisioning ...                              context=docker
INFO[0040] starting ...                                  context=docker
INFO[0046] done                                         

colima list
PROFILE    STATUS     ARCH       CPUS    MEMORY    DISK     RUNTIME    ADDRESS
default    Running    aarch64    2       2GiB      60GiB    docker

I am running macOS Monterey 12.4 on an M1 Mac.

DezzardHD commented 1 year ago

I managed to get it working three weeks ago using the instructions posted by codrutP.

Previously, I had tried using Colima by installing Docker, Docker Compose, and then Colima via Homebrew. Starting Colima resulted in the following error message:

❯ colima start --edit oraxe
INFO[0000] editing in vim
INFO[0023] starting colima [profile=oraxe]
INFO[0023] runtime: docker
INFO[0023] preparing network ...                         context=vm
öWARN[0038] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status oraxe], output: "time=\"2023-04-25T16:50:13+02:00\" level=fatal msg=\"pid file not found: stat /Users/user/.colima/oraxe/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0038] disk size cannot be reduced, ignoring...      context=vm
WARN[0038] error setting up reachable IP address: vmnet socket file not found: stat /Users/user/.colima/oraxe/daemon/vmnet.sock: no such file or directory
INFO[0038] starting ...                                  context=vm
> [hostagent] Shutting down the host agent
> [hostagent] failed to exit SSH master
> [hostagent] Shutting down QEMU with ACPI
> [hostagent] failed to open the QMP socket "/Users/user/.lima/colima-oraxe/qmp.sock", forcibly killing QEMU
> [hostagent] QEMU has already exited
> exiting, status={Running:false Degraded:false Exiting:true Errors:[] SSHLocalPort:0} (hint: see "/Users/user/.lima/colima-oraxe/ha.stderr.log")
FATA[0039] error starting vm: error at 'starting': exit status 1

After that, I followed the above-mentioned instructions, which led to:

❯ colima start --edit oraxe
INFO[0000] editing in vim
colima is currently running, restart to apply changes? [y/N] y
INFO[0051] stopping colima [profile=oraxe]
INFO[0051] stopping ...                                  context=docker
INFO[0052] stopping ...                                  context=vm
INFO[0057] done
INFO[0061] starting colima [profile=oraxe]
INFO[0061] runtime: docker
INFO[0061] preparing network ...                         context=vm
WARN[0076] error starting network: error at 'preparing network': error running [/opt/homebrew/bin/colima daemon status oraxe], output: "time=\"2023-04-25T17:02:38+02:00\" level=fatal msg=\"pid file not found: stat /Users/user/.colima/oraxe/daemon/daemon.pid: no such file or directory\"", err: "exit status 1"  context=vm
WARN[0076] disk size cannot be reduced, ignoring...      context=vm
WARN[0076] error setting up reachable IP address: vmnet socket file not found: stat /Users/user/.colima/oraxe/daemon/vmnet.sock: no such file or directory
INFO[0076] starting ...                                  context=vm
INFO[0128] provisioning ...                              context=docker
INFO[0128] starting ...                                  context=docker
INFO[0134] done

Three weeks ago, I was confused about the warning ('error setting up reachable IP address: [...]'), thinking that my Docker containers wouldn't function because of it. However, it turned out that they functioned without any problems.

Today, I reinstalled Colima due to an issue running docker login. For some reason, Docker was unable to connect to Docker Hub for authentication. After a fresh install of Colima, I was able to resolve my issue with docker login. I don't know how to recreate the problem.

Regarding issue #458: I would like colima to work right out of the box when installing it via brew install. Instead I had to follow the above mentioned instructions.

PS: I used --edit instead of --network-address for setting network.address to true.

OksiBlack commented 1 year ago

For me the issue persists M1 Ventura, latest colima version

WARN[0181] error setting up reachable IP address: vmnet socket file not found: stat /.../.colima/default/daemon/vmnet.sock: no such file or directory

Instructions havent changed the result.

ejames17 commented 1 year ago

Same for me @OksiBlack

madpah commented 1 year ago

Certainly not the fix, but this works as a workaround for me when colima status shows no Network Address:

  1. Start your colima VM with whatever command you are using (ensuring you include --network-address option (Wait for it to start) I confirm it's started with no network address by running:

    > colima status
INFO[0000] colima is running using macOS Virtualization.Framework
INFO[0000] arch: aarch64
INFO[0000] runtime: docker
INFO[0000] mountType: virtiofs
INFO[0000] address:
INFO[0000] socket: unix:///Users/username/.colima/default/docker.sock

  2. On my Mac, there should now be a network bridge adapter created - mine is always bridge100 - grab it's IP address by running:

    > ifconfig bridge100 | grep "inet "
inet 192.168.106.1 netmask 0xffffff00 broadcast 192.168.106.255

  3. colima ssh

(you are now in SSH inside VM)

  1. sudo su -
  2. Edit the file /etc/network/interfaces so that it looks similar to the below, using the IP address returned in step 2 as the value for gateway: 
    
auto lo
iface lo inet loopback

auto eth0 iface eth0 inet dhcp

auto col0 iface col0 inet static address 192.168.106.2 netmask 255.255.255.0 gateway 192.168.106.1

6. Run `ifdown col0`
7. Run `ifup col0`
8. Exit ssh in colima by pressing `CTRL + D` twice

This results in the VM having the IP `192.168.106.2` - validated by again running `colima status`:

colima status INFO[0000] colima is running using macOS Virtualization.Framework INFO[0000] arch: aarch64 INFO[0000] runtime: docker INFO[0000] mountType: virtiofs INFO[0000] address: 192.168.106.2 INFO[0000] socket: unix:///Users/username/.colima/default/docker.sock

ejames17 commented 1 year ago

@madpah Thanks for the workaround. I did a factory reset and its been hold steady for now. Some others using the trellis project have similar issues. Ill refer back here.

foxted commented 1 year ago

Trying all the solution suggested above, still having that issue. Though I tried limactl start and it can boot without issues. I tried the command I found in daemon.log and it works as well, but colima start --network-address does not:

Here's my daemon.log: 

time="2023-08-29T14:19:36-07:00" 
level=info msg="- - - - - - - - - - - - - - -"
time="2023-08-29T14:19:36-07:00" level=info msg="daemon started by colima"
time="2023-08-29T14:19:36-07:00" level=info msg="Run `/usr/bin/pkill -F /Users/t944521/.colima/default/daemon/daemon.pid` to kill the daemon"
time="2023-08-29T14:19:36-07:00" level=trace msg="cmd int [\"sudo\" \"/opt/colima/bin/socket_vmnet\" \"--vmnet-mode\" \"shared\" \"--socket-group\" \"staff\" \"--vmnet-gateway\" \"192.168.106.1\" \"--vmnet-dhcp-end\" \"192.168.106.254\" \"--pidfile\" \"/opt/colima/run/vmnet-default.pid\" \"/Users/t944521/.colima/default/daemon/vmnet.sock\"]"
sudo: a terminal is required to read the password; either use the -S option to read from standard input or configure an askpass helper
sudo: a password is required
time="2023-08-29T14:19:36-07:00" level=error msg="error starting vmnet: error running vmnet: exit status 1"
time="2023-08-29T14:19:36-07:00" level=info msg="terminate signal received"
time="2023-08-29T14:19:36-07:00" level=fatal msg="context canceled"

limactl start colima yields: 

INFO[0000] Using the existing instance "colima"
INFO[0000] QEMU binary "/opt/homebrew/bin/qemu-system-aarch64" seems properly signed with the "com.apple.security.hypervisor" entitlement
INFO[0000] [hostagent] Starting QEMU (hint: to watch the boot progress, see "/Users/t944521/.lima/colima/serial*.log")
INFO[0000] SSH Local Port: 52644
INFO[0000] [hostagent] Waiting for the essential requirement 1 of 5: "ssh"
INFO[0021] [hostagent] The essential requirement 1 of 5 is satisfied
INFO[0021] [hostagent] Waiting for the essential requirement 2 of 5: "user session is ready for ssh"
INFO[0021] [hostagent] The essential requirement 2 of 5 is satisfied
INFO[0021] [hostagent] Waiting for the essential requirement 3 of 5: "sshfs binary to be installed"
INFO[0021] [hostagent] The essential requirement 3 of 5 is satisfied
INFO[0021] [hostagent] Waiting for the essential requirement 4 of 5: "/etc/fuse.conf (/etc/fuse3.conf) to contain \"user_allow_other\""
INFO[0021] [hostagent] The essential requirement 4 of 5 is satisfied
INFO[0021] [hostagent] Waiting for the essential requirement 5 of 5: "the guest agent to be running"
INFO[0022] [hostagent] The essential requirement 5 of 5 is satisfied
INFO[0022] [hostagent] Mounting "/Users/t944521" on "/Users/t944521"
INFO[0022] [hostagent] Mounting "/tmp/colima" on "/tmp/colima"
INFO[0022] [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
INFO[0022] [hostagent] Forwarding "/var/run/docker.sock" (guest) to "/Users/t944521/.colima/default/docker.sock" (host)
INFO[0022] [hostagent] Forwarding "/var/run/docker.sock" (guest) to "/Users/t944521/.colima/docker.sock" (host)
INFO[0022] [hostagent] The final requirement 1 of 1 is satisfied
INFO[0022] READY. Run `limactl shell colima` to open the shell.
jamhed commented 1 year ago

Hello guys,

To whom it may concern:

colima version 0.5.5, arch: aarch64, macbook air m2, vz/virtiofs

With --network-address option set, colima starts a virtual machine, and bridges col0 interface with bridge100, which has ip address 192.168.106.1. It seems that bridge100 interface is a special one used for "internet sharing", which means that there should be a DHCP server running. However, if for some reason "internet sharing" is disabled, then there is no DHCP server on bridge100 interface, and therefore, lima's virtual machine is unable to acquire IP address, and therefore k3s kubernetes fails to start, which causes colima error "The connection to the server 127.0.0.1:6443 was refused - did you specify the right host or port?"

The solution is to start dhcp server on bridge100 serving 192.168.106.* range, e.g., dnsmasq, which might require disabling macos builtin server bootps.plist, due to port conflict.

tzookb commented 11 months ago

@jamhed can you share what commands we actually need?

or any other solutions?