getsentry / devenv

unified dev environment management tool
4 stars 2 forks source link

colima and lima upgrade #146

Open joshuarli opened 1 week ago

joshuarli commented 1 week ago

https://github.com/abiosoft/colima/issues/949#issuecomment-2317569867

the double-start issue has since been fixed - we had a workaround for this before, but it would be good to upgrade past that

it must be noted that lima >=0.23.2 is necessary (see https://github.com/lima-vm/lima/issues/2556)

joshuarli commented 2 days ago

colima 0.7.5 changed the docker socket location (at least for me, since it now uses XDG_CONFIG_HOME, and ~/.colima is removed by us during upgrades), will need to make it conform to the previous ~/.colima default (otherwise devservices AND peoples docker contexts will break)

after fixing that, it isn't happy with lima 0.19.1 anymore - we still detect colima's unhealthy (after initial start) then the 2nd start fails: error starting docker: cannot restart, VM not previously started

joshuarli commented 2 days ago

colima v0.7.5 + limactl 0.23.2 on Macos 14.5 gets past that^ as the double start problem is fixed

but then results in

INFO[0000] creating and starting ...                     context=vm
> Terminal is not available, proceeding without opening an editor
> `vmType: vz` is experimental
> Starting the instance "colima" with VM driver "vz"
> "Attempting to download the image" arch=aarch64 digest= location=/Users/josh/.cache/colima/caches/fc0bfb9256e1f61294f2c41fc95ff4feb6c0daafccfb3b435e2d972e39bc8432.raw
> Downloaded the image from "/Users/josh/.cache/colima/caches/fc0bfb9256e1f61294f2c41fc95ff4feb6c0daafccfb3b435e2d972e39bc8432.raw"
> Converting "/Users/josh/.colima/_lima/colima/basedisk" (raw) to a raw disk "/Users/josh/.colima/_lima/colima/diffdisk"
> Expanding to 60GiB
> [hostagent] hostagent socket created at /Users/josh/.colima/_lima/colima/ha.sock
> [hostagent] Starting VZ (hint: to watch the boot progress, see "/Users/josh/.colima/_lima/colima/serial*.log")
> [hostagent] Setting up Rosetta share
> [hostagent] Unable to configure Rosetta: failed to create a new rosetta directory share caching option: unsupported build target macOS version for 14.0 (the binary was built with __MAC_OS_X_VERSION_MAX_ALLOWED=130100; needs recompilation)
> SSH Local Port: 56329
> [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
> [hostagent] [VZ] - vm state change: running
> [hostagent] Waiting for the essential requirement 1 of 2: "ssh"
> [hostagent] The essential requirement 1 of 2 is satisfied
> [hostagent] Waiting for the essential requirement 2 of 2: "user session is ready for ssh"
> [hostagent] The essential requirement 2 of 2 is satisfied
> [hostagent] Waiting for the guest agent to be running
> [hostagent] Forwarding "/var/run/docker.sock" (guest) to "/Users/josh/.colima/default/docker.sock" (host)
> [hostagent] Forwarding "/var/run/docker.sock" (guest) to "/Users/josh/.colima/docker.sock" (host)
> [hostagent] Guest agent is running
> [hostagent] Waiting for the final requirement 1 of 1: "boot scripts must have finished"
> [hostagent] Not forwarding TCP 127.0.0.53:53
> [hostagent] Not forwarding TCP 127.0.0.54:53
> [hostagent] Not forwarding TCP [::]:22
> [hostagent] The final requirement 1 of 1 is satisfied
> READY. Run `limactl shell colima` to open the shell.
> stat: cannot statx '/proc/sys/fs/binfmt_misc/rosetta': No such file or directory
> sh: 1: echo: echo: I/O error
WARN[0012] unable to enable rosetta: exit status 1

This isn't an issue with ensuring rosetta's running, it's an issue with lima (https://github.com/lima-vm/lima/issues/1182)

Only becomes a problem when we try and emulate snuba (sync doesn't cover that, running migrations doesn't require snuba)

joshuarli commented 2 days ago

using a locally compiled limactl 0.23.2 works

homebrew works as well /opt/homebrew/Cellar/lima/0.23.2/bin/limactl, i think we should prefer homebrew binaries to github releases

https://formulae.brew.sh/api/formula/lima.json https://ghcr.io/v2/homebrew/core/lima/blobs/sha256:be8e2b92961eca2f862f1a994dbef367e86d36705a705ebfa16d21c7f1366c35

unfortunately you need to use QQ== bearer token (or get a legit anonymous one from https://ghcr.io/token?service=ghcr.io&scope=repository%3Ahomebrew/core/go%3Apull)

structure of brew's tgz blob is slightly different (lima/0.23.2 prefix)