search

bottlerocket-os / bottlerocket-admin-container

A container for admin access to Bottlerocket
Other
58 stars 34 forks source link

admin container should run on hosts with cgroup v2 enabled #75

Closed markusboehme closed 12 months ago

markusboehme commented 1 year ago

Image I'm using: any Bottlerocket variant, any admin container image

Issue or Feature Request:

The admin container should be able to run on Bottlerocket hosts that use cgroup v2/the unified cgroup hierarchy. Currently, the admin container cannot start on such hosts and will exit with the following errors:

[   15.511906] host-ctr[1189]: Cannot determine cgroup we are running in: No such file or directory
[   15.512878] host-ctr[1189]: Failed to allocate manager object: No such file or directory
[   16.027735] host-ctr[1189]: time="2023-01-11T15:49:06Z" level=fatal msg="Container admin exited with non-zero status"

The reason is Amazon Linux 2 in the base image shipping with systemd v219. This version of systemd does not support cgroup v2.

markusboehme commented 1 year ago

With the approach in #76 being insufficient for non-superpowered admin containers that cannot mount filesystems on their own, another approach is needed.

Untested idea: Some of the compatibility work spills over into host-ctr that mounts a cgroup v1 root in /sys/fs/cgroup on demand instead of the default bind mount of the host's /sys/fs/cgroup. The functionality could hide behind a flag for host-ctr that is passed by host-containers exclusively for the admin container.

The elephant in the room is the unattractive special-casing of the admin container. It could be removed once the admin container image moves to a base image that supports cgroup v2. Alternatively, the host-ctr flag could generically be exposed via the settings model, but there would be no good choice for a default that fits both now and in a cgroup v2-only future.

markusboehme commented 1 year ago

Untested idea: Some of the compatibility work spills over into host-ctr that mounts a cgroup v1 root in /sys/fs/cgroup on demand instead of the default bind mount of the host's /sys/fs/cgroup. The functionality could hide behind a flag for host-ctr that is passed by host-containers exclusively for the admin container.

After hacking a bit on this and looking at code of containerd and runc, this appears to not be a great idea. cgroup filesystems are understandably special-cased in both of them.

markusboehme commented 12 months ago

Cleaning up loose ends: #76 was merged (see more discussion there) and released with v0.10.0.