search

kubernetes-sigs / kind

Kubernetes IN Docker - local clusters for testing Kubernetes
https://kind.sigs.k8s.io/
Apache License 2.0
13.42k stars 1.55k forks source link

Cannot start with podman backend due to AppArmor errors #1757

Closed jbarrick-mesosphere closed 4 years ago

jbarrick-mesosphere commented 4 years ago

What happened:

I cannot start kind using the podman provider, due to an error indicating AppArmor is not enabled:

Error: Apparmor profile "unconfined" specified, but Apparmor is not enabled on this system

What you expected to happen:

The kind cluster to start.

How to reproduce it (as minimally and precisely as possible):

➜  ~ sudo aa-status
apparmor module is loaded.
48 profiles are loaded.
48 profiles are in enforce mode.
   /usr/lib/apache2/mpm-prefork/apache2
   /usr/lib/apache2/mpm-prefork/apache2//DEFAULT_URI
   /usr/lib/apache2/mpm-prefork/apache2//HANDLING_UNTRUSTED_INPUT
   /usr/lib/apache2/mpm-prefork/apache2//phpsysinfo
   /usr/lib/dovecot/anvil
   /usr/lib/dovecot/auth
   /usr/lib/dovecot/config
   /usr/lib/dovecot/deliver
   /usr/lib/dovecot/dict
   /usr/lib/dovecot/dovecot-auth
   /usr/lib/dovecot/dovecot-lda
   /usr/lib/dovecot/dovecot-lda//sendmail
   /usr/lib/dovecot/imap
   /usr/lib/dovecot/imap-login
   /usr/lib/dovecot/lmtp
   /usr/lib/dovecot/log
   /usr/lib/dovecot/managesieve
   /usr/lib/dovecot/managesieve-login
   /usr/lib/dovecot/pop3
   /usr/lib/dovecot/pop3-login
   /usr/lib/dovecot/ssl-params
   /usr/lib/dovecot/stats
   /usr/sbin/dnsmasq
   /usr/sbin/dnsmasq//libvirt_leaseshelper
   apache2
   apache2//DEFAULT_URI
   apache2//HANDLING_UNTRUSTED_INPUT
   apache2//phpsysinfo
   avahi-daemon
   docker-default
   dovecot
   identd
   klogd
   lsb_release
   mdnsd
   nmbd
   nscd
   ntpd
   nvidia_modprobe
   nvidia_modprobe//kmod
   ping
   smbd
   smbldap-useradd
   smbldap-useradd///etc/init.d/nscd
   syslog-ng
   syslogd
   traceroute
   winbindd
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.
➜  ~ sudo KIND_EXPERIMENTAL_PROVIDER=podman kind create cluster --name newcluster
using podman due to KIND_EXPERIMENTAL_PROVIDER
enabling experimental podman provider
Creating cluster "newcluster" ...
 ✓ Ensuring node image (kindest/node:v1.18.2) 🖼
 ✗ Preparing nodes 📦 
ERROR: failed to create cluster: podman run error: command "podman run --hostname newcluster-control-plane --name newcluster-control-plane --label io.x-k8s.kind.role=control-plane --privileged --security-opt seccomp=unconfined --security-opt apparmor=unconfined --tmpfs /tmp --tmpfs /run --volume /var --volume /lib/modules:/lib/modules:ro --detach --tty --label io.x-k8s.kind.cluster=newcluster --publish=127.0.0.1:40923:6443/TCP kindest/node@sha256:7b27a6d0f2517ff88ba444025beae41491b016bc6af573ba467b70c5e8e0d85f" failed with error: exit status 125
Command Output: WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
WARNING: The same type, major and minor should not be used for multiple devices.
Error: Apparmor profile "unconfined" specified, but Apparmor is not enabled on this system
➜  ~

Environment:

BenTheElder commented 4 years ago

podman 2.X was worked around in HEAD, recently, can you try that?

BenTheElder commented 4 years ago

https://github.com/kubernetes-sigs/kind/pull/1728

mlouage commented 4 years ago

So for Podman 2.x I cannot use a kind download from the releases? I downloaded the latest release (0.8.1) and I also have this issue.

Fedora 32 with Podman 2.0.4 (rootless). I have no App Armor installed, I believe Fedora has SELinux which should be enabled on my system.

BenTheElder commented 4 years ago

You can't, podman broke handling of the app armor flag relative to docker. It should be worked around if you install from the latest kind sources, but I can't confirm that myself.

BenTheElder commented 4 years ago

Podman 2.X has multiple breaking changes kind is working around in https://github.com/kubernetes-sigs/kind/pull/1728

Fedora 32 with Podman 2.0.4 (rootless). I have no App Armor installed, I believe Fedora has SELinux which should be enabled on my system.

Rootless podman does not work and has never worked. You will need to use it rootfully to run Kubernetes for now.

Rootless Kubernetes does not exist upstream yet, but even when it does rootless podman has it's own issues. See discussion in #1727.

mlouage commented 4 years ago

Ok, thanks. I'll try to build it from source. Not too familiar with Go but I'll give it a try!

BenTheElder commented 4 years ago

you can build from source with make build from a clone without installing go, (it will use docker/podman), the binary will be in bin/kind

BenTheElder commented 4 years ago

v0.9.0 is slated for tomorrow tentatively, this should be fixed already. cgroupsv2 is still pending some other work, mostly not in the kind repo.