Noble uses control groups (cgroups) v2 instead of v1

[cunnie]

Control Groups v2 is the new standard; Ubuntu switched to it as the default since pre-Jammy Impish Indra (21.10).

Kernel 6.5 + cgroups v1 has caused problems (OOM during staging): #318

This change will probably affect the following components:

• TAS
  • bpm
  • runc
  • garden
• TKGi

[rkoster]

Garden cgroups v2 support: https://github.com/cloudfoundry/guardian/commit/a11a929395980a7ccd5d44ad7bc68ae70ff350e5

[cunnie]

If you're testing on a Jammy stemcell with cgroups v2, and the agent isn't coming up ("unresponsive agent"), then you need to do the following:

• you need to do ssh onto the VM (yes, I know, the tricky part)
• then flush the iptables mangle table iptables -F -t mangle, which has a rule inserted by the BOSH Agent which restricts who can talk to the NATS — only if they're in a v1 cgroup, but there are no v1 cgroups, so Agent can't talk to the NATS.
• and, while you're at it, you might want to replace /var/vcap/bosh/etc/monit-access-helper.sh's permit_monit_access() with a ::

-    net_cls_location="$(cat /proc/self/mounts | grep ^cgroup | grep net_cls | awk '{ print $2 }' )"
-    monit_access_cgroup="${net_cls_location}/monit-api-access"
-
-    mkdir -p "${monit_access_cgroup}"
-    echo "${monit_isolation_classid}" > "${monit_access_cgroup}/net_cls.classid"
-
-    echo $$ > "${monit_access_cgroup}/tasks"
+       :
 }

Note: we need to get rid of monit-access-helper.sh because hopefully we'll have deprecated monit by the time we get to Noble.

Also, see above for where in the BOSH Agent we need to make changes to accommodate cgroups-v1-with-monit and cgroups-v2-no-monit.

[ramonskie]

current noble stemcell is on cgroup v2 https://github.com/cloudfoundry/bosh-linux-stemcell-builder/commit/933220bb3c24781bd6e2d983793c570c2add1940

currently i have removed the helpers and monit wrapper and used the following iptable rules

        -m cgroup \! --path "/system.slice/monit.service" -j DROP
then
  /bin/true
else
    iptables -t mangle -I POSTROUTING -d 127.0.0.1 -p tcp --dport 2822 \
         -m cgroup \! --path "/system.slice/monit.service" -j DROP
    iptables -t mangle -I POSTROUTING -d 127.0.0.1 -p tcp --dport 2822 \
         -m state --state ESTABLISHED,RELATED -j ACCEPT
fi

[ramonskie]

we moved to nftables see https://github.com/cloudfoundry/bosh-linux-stemcell-builder/commit/e1245f41aa5db4acc6d7e3bc81acab0ebdd8509f

[ramonskie]

noble now uses cgroups v2. there is a issue created for warden #352