Upgrade to v0.21.0-k3s1r0, get CreateContainerError on all pods

[MaximizeRr]

Version (k3OS / kernel) k3os version v0.21.0-k3s1r0 5.4.0-72-generic rancher/k3os#80 SMP Tue May 4 19:31:44 UTC 2021

Architecture x86_64

Describe the bug After updating from v0.20.6-k3s1r0 to v0.21.0-k3s1r0, i get CreateContainerError on all pod. In the k8s events i get this error message:

Error: failed to create containerd container: get apparmor_parser ver
sion: exec: "apparmor_parser": executable file not found in $PATH

To Reproduce Upgrade a single master node from v0.20.6-k3s1r0 to v0.21.0-k3s1r0 by changing k3os-latest plan.

kubectl edit plan k3os-latest -n k3os-system
kubectl label nodes yournode k3os.io/upgrade=enabled 

Expected behavior Pods in running status

Actual behavior No container can be start on the cluster. Example:

kube-system            traefik-6f9cbd9bd4-xhmmq                                          0/1     CreateContainerError        0          31m
kube-system            local-path-provisioner-5ff76fc89d-zmtnv                           0/1     CreateContainerError        0          31m
kube-system            metrics-server-86cbb8457f-sh2vg                                   0/1     CreateContainerError        0          31m
k3os-system            system-upgrade-controller-8bf4f84c4-gfmx4                         0/1     CreateContainerError        0          15h
kube-system            coredns-7448499f4d-z2hl2                                          0/1     CreateContainerError        0          29m

[brlbil]

I have the same issue with a freshly installed cluster, also created issue for k3s

[dweomer]

This is an issue arising from the fact that k3OS ships a repackaged Ubuntu kernel which has apparmor enabled by default but lacks apparmor userspace tooling combined with the fact that upstream kubelet in 1.21.x detects apparmor via kernel and attempts to enable profile(s). The solution for k3OS is to update your boot parameters, a la https://github.com/rancher/k3os/pull/695/commits/58564fea3de1a2d2d8425689216da71e6935e632, to disable apparmor.

[smshr]

I got the same error with k3s version v1.21.1+k3s1 running in debian 10.9, reverting back to the previous version v1.20.7+k3s1 fixed it.

Error: failed to create containerd container: get apparmor_parser version: exec: "apparmor_parser": executable file not found in $PATH

[brlbil]

@dweomer I made a new installation with the grub file that has apparmor=0 boot parameter. I can verify that solved the issue.

[dweomer]

I got the same error with k3s version v1.21.1+k3s1 running in debian 10.9, reverting back to the previous version v1.20.7+k3s1 fixed it.

Error: failed to create containerd container: get apparmor_parser version: exec: "apparmor_parser": executable file not found in $PATH

(for debian you should either disable apparmor at kernel boot as suggested here or install the apparmor userland)

[smshr]

(for debian you should either disable apparmor at kernel boot as suggested here or install the apparmor userland)

installing apparmor userland fixed it apt install apparmor apparmor-utils

[dweomer]

This should be fixed in the next 0.21.x release

[belachkar]

Fixed by installing apparmor-parser. OpenSUSE cmd: zypper install apparmor-parser

[haikalshiddiq]

Fixed by installing apparmor-parser.

thank you bro, it fixes my issue on Vagrant using OpenSUSE 15.2 :)

[rahul-aryan]

@belachkar @haikalshiddiq I am using Opensuse 15.2 on vagrant ...please tell me which command to use to fix this issue ?

[belachkar]

@rahul-aryan zypper install apparmor-parser

[Eeebru]

I'm facing the same issue, using OpenSUSE 15.2 on vagrant, zypper install apparmor-parser worked

[marksarchi]

Same issue with opensuse/Leap-15.2.x86_64 on vagrant , zypper install solved the issue

[jory3]

same issue with Photon OS 4. Solved by installing apparmor-parser:

tdnf install apparmor-parser