lima-vm / alpine-lima

Create an alpine based image for lima
Apache License 2.0
40 stars 26 forks source link

Enable setting cgroup v2 mode in std edition #114

Closed na0x2c6 closed 7 months ago

na0x2c6 commented 9 months ago

Hello.

I previously mentioned briefly about enabling cgroup v2 in the Alpine image at https://github.com/lima-vm/lima/discussions/1878. I have made modifications to allow changing the mode through environment variables during machine startup.

The default mode is hybrid mode, so I believe it does not affect the existing functionality. I would appreciate it if you could consider incorporating this modification if possible.

This modification was made with reference to the comment I received at https://github.com/lima-vm/lima/discussions/1878#discussioncomment-7177525. Thank you!

The pre-built image can be found at the following location:

https://github.com/na0x2c6/alpine-lima/releases/tag/v0.2.32-std-cgroup-experimental

The lima template that is currently being used is available here.

Thanks.

na0x2c6 commented 9 months ago

I apologize for requesting a re-review without making any modifications. I commented because I failed to adequately explain the intent of the modifications. I would appreciate it if you could check it.

https://github.com/lima-vm/alpine-lima/pull/114#discussion_r1399180560

jandubois commented 8 months ago

Sorry, this PR somehow dropped from my radar. But looking at it again, don't you get the same effect from:

provision:
- mode: system
  script: |
    echo rc_cgroup_mode=unified >/etc/conf.d/cgroups
    rc-service cgroups restart

In which case this PR wouldn't be needed. Or am I missing something again?

na0x2c6 commented 8 months ago

@jandubois

Thank you for your reply.

The scripts defined in the lima template are applied after mounting the cgroup here, so the cgroup v1 will be mounted on the first boot. Once it is mounted, it is not easy to unmount it because some processes are started with cgroup v1:

$ ls /sys/fs/cgroup/*/*/tasks
/sys/fs/cgroup/openrc/acpid/tasks  /sys/fs/cgroup/openrc/lima-guestagent/tasks  /sys/fs/cgroup/openrc/sshd/tasks

# show PIDs in cgroup v1
$ cat /sys/fs/cgroup/*/*/tasks
2418
2594
2596
2602
...

In other words, once cgroup v1 is mounted, rc-service cgroups restart cannot apply cgroup v2, thus necessitating a machine restart.

( https://github.com/lima-vm/lima/discussions/1878 is a question about whether there is a way to bypass the above problem. )

Actually, I wanted to use cgroup v2 because I am using rootless podman. Also, I have been using the alpine template because I couldn't start Fedora with the vmType: "vz" configuration. However, thanks to the great fixes in lima v0.19, such as https://github.com/lima-vm/lima/pull/2026, https://github.com/lima-vm/lima/pull/1926, https://github.com/lima-vm/lima/pull/1942, https://github.com/lima-vm/lima/pull/1965, I can now comfortably use the podman template.

So, to be honest, there is currently not much motivation to use cgroup v2 in the alpine image. If this PR seems helpful for future purposes, I would be happy to make the necessary modifications. However, if there is no specific reason to incorporate it, I would like to close it.

I am grateful for the development of this excellent image. Thank you for taking the time to review.

jandubois commented 7 months ago

Thank you for your contribution, but I don't want to apply this PR (anymore).

Alpine 3.19 defaults to using cgroup v2.

It is possible to revert back to v1 by editing /etc/rc.conf and rebooting in a provisioning script. This takes an extra 5-10s because it happens early during boot, and is only needed when you want to switch cgroup modes. I don't expect anyone wanting to switch back to hybrid mode, now that unified mode is working pretty well on Alpine.

I believe this is good enough. Please let me know if you disagree!

jandubois commented 7 months ago

I don't expect anyone wanting to switch back to hybrid mode

Well, actually, that was a bit hasty: In Rancher Desktop we switch back to hybrid mode when the user runs an older Kubernetes version because k3s only started supporting cgroup v2 in 1.20.4. So these are ancient/unsupported k3s versions, but they will still run on Rancher Desktop 1.13 using Alpine 3.19.

We select the cgroup mode with an environment variable in lima.yaml, like

env:
  RC_CGROUP_MODE: hybrid

And use this provisioning script to switch, when necessary: https://github.com/rancher-sandbox/rancher-desktop/blob/62807cbc4cff6c1099b966b6496f9770e050cb93/pkg/rancher-desktop/assets/lima-config.yaml#L41-L53

na0x2c6 commented 7 months ago

Alpine 3.19 defaults to using cgroup v2.

I understand. I don't think it's a problem at all either. Thank you for confirming my issue/PR.