Update docs to describe how the underlying VM and tools can be updated

[spkane]

nerdctl has a new release: https://github.com/containerd/nerdctl/releases/tag/v0.15.0

It would be nice to add some documentation and even a helper script on how users can update some of the core CLI binaries like this, since some of them are in very active development.

• check the nerdctl version

colima nerdctl version

• enter the VM

colima ssh

• Update the binary

cd /usr/local
sudo wget https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-full-0.15.0-linux-amd64.tar.gz
sudo tar -xvf ./nerdctl-full-0.15.0-linux-amd64.tar.gz
exit

• check the nerdctl version

colima nerdctl version

[jandubois]

Typically there is a new lima release shortly after any nerdctl release, so all you have to do is update lima once it is out.

If you want to upgrade nerdctl before, then the best way is probably to create a $HOME/.lima/_config/override.yaml file:

containerd:
  archives:
  - location: "https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-full-0.15.0-linux-amd64.tar.gz"
    arch: "x86_64"
    digest: "sha256:..."

You'll have to lookup the digest yourself... 😄

You can check the builtin default values like this:

$ limactl info | jq .defaultTemplate.containerd.archives
[
  {
    "location": "https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-full-0.15.0-linux-amd64.tar.gz",
    "arch": "x86_64",
    "digest": "sha256:ca40d99d257e69f0220bb1cbdab1b602032692f45f713c901f328d2f4e3c12b3"
  },
  {
    "location": "https://github.com/containerd/nerdctl/releases/download/v0.15.0/nerdctl-full-0.15.0-linux-arm64.tar.gz",
    "arch": "aarch64",
    "digest": "sha256:dd8639ce868bab394467576f55375c4b40a8288badb579d2e30c3487da6004ea"
  }
]

Oh, and you can copy the digest value from there (I'm running latest lima dev branch).

[jandubois]

If you want to upgrade nerdctl before, then the best way is probably to create a $HOME/.lima/_config/override.yaml file:

Don't forget to remove/edit the override once you update lima; otherwise you have pinned your nerdctl version forever...

[jandubois]

I just double-checked, and lima 0.8.0 is available on brew, and uses nerdctl 0.15.0, so upgrading lima and restarting your VM should be all you need.

[spkane]

Ahh. Great. That makes sense. Thanks!

[spkane]

Adding this to the docs could still be a plus, since it might not be obvious how all the parts interact.

[abiosoft]

Since it appears inevitable that users will not be sufficed with the autogenerated config, I will consider adding this to docs. Other than the image iso (and maybe the ssh port), every other config can be overridden without issues.

I will also consider adding a flag to accommodate additional provision scripts.

[jandubois]

I will also consider adding a flag to accommodate additional provision scripts.

Not sure why you would need a flag; any provisioning scripts from override.yaml will be added before the scripts in lima.yaml, and provisioning scripts from default.yaml will be added after.

The "merge" rules for lists and maps are not straight replacements, and the rules may vary for different properties, to maximize utility of the default/override feature. This is (tersely) documented at the end of default.yaml.

[abiosoft]

Not sure why you would need a flag; any provisioning scripts from override.yaml will be added before the scripts in lima.yaml, and provisioning scripts from default.yaml will be added after.

That's a good point, I did not think of that.

Thanks.

[jandubois]

@abiosoft I have not checked if you have implemented support for LIMA_HOME now, but it would be useful because default/override settings affect all instances under the same LIMA_HOME directory.

So I think it would be good to have COLIMA_HOME just for colima, and internally set LIMA_HOME=$COLIMA_HOME before calling out to limactl. That way you can have lima and colima coexist peacefully, and the colima overrides don't affect the rest of your lima instances.

It would just be an option. If COLIMA_HOME isn't set, you would just pass-through LIMA_HOME. Anyways, just a suggestion.

PS: medium/long term I want to add a per-instance override mechanism, but don't know when I will have time for this.

[abiosoft]

@abiosoft I have not checked if you have implemented support for LIMA_HOME now, but it would be useful because default/override settings affect all instances under the same LIMA_HOME directory.

I have implemented support for LIMA_HOME, yeah. Though I have not done similar for colima so I will plan that for the next release.

PS: medium/long term I want to add a per-instance override mechanism, but don't know when I will have time for this.

This was another reason I thought of provision script support. If I have time as well, I can take that on upstream.

[jandubois]

If I have time as well, I can take that on upstream.

Please discuss first before starting the work. I want this to integrate with manually changing config settings, e.g. limactl start myinstance --cpus 8. But that means that the file would also be modified by limactl, so e.g. comments would get lost.

Also, when you delete an instance, any override stored inside the instance directory would be lost as well, even though I may have wanted to recreate a pristine copy right after deleting the old one.

I haven't figured out how the pieces should all fit together, which is the main reason I haven't started on this already (you can see some very old discussion about this in https://github.com/lima-vm/lima/issues/131).

[myannou]

I hope it is the right thread to post this question related on how to override provision scripts.

I tried to add some overrides to the colima vm in ~/.lima/_config/override.yaml :

provision:
- mode: system
  script: |
    #!/bin/sh
    set -eux -o pipefail
    echo "create alias k for kubectl on system mode"
    alias k=kubectl

- mode: user
  script: |
    #!/bin/sh
    set -eux -o pipefail
    echo "create alias k for kubectl on user mode"
    alias k=kubectl

but when I start the colima vm colima start --with-kubernetes --runtime containerd then the command colima ssh -- sudo k get pod -A or colima ssh -- k get pod -A return:

/bin/ash: k: not found
FATA[0000] exit status 127

I checked in the vm colima ssh -- sudo cat /mnt/lima-cidata/provision.system/00000000 contains:

#!/bin/sh
set -eux -o pipefail
echo "create alias k for kubectl on system mode"
alias k=kubectl

but it doesn't seem to be executed and for the "user provision", the directory "/home/${LIMA_CIDATA_USER}.linux/.lima-user-script" is empty

Am I missing something ? Thanks for your help

[abiosoft]

@myannou why do you need to set alias in the VM?

Firstly, aliases do not span shell sessions, so the alias would not be present. Except you set it in an equivalent of .bashrc for ash.

Kubernetes client kubectl is on your host, you should have the alias on the host instead inside .zshrc for zsh or .bashrc for bash.

[myannou]

@myannou why do you need to set alias in the VM?

it was just a test (probably not appropriated) to understand how to add scripts/packages/... in the colima vm. what I would like to achieve is to automatically install an ingress controller on k3s when starting a new colima vm with kubernetes... If I do it manually it looks like:

helm repo update
helm upgrade --install ingress-nginx ingress-nginx --repo https://kubernetes.github.io/ingress-nginx --namespace ingress-nginx --create-namespace --set controller.service.nodePorts.http=30080 --set controller.service.nodePorts.https=30443

So is it the right approach to use "~/.lima/_config/override.yaml " with the "provision" section ? Thanks