SELinux Trouble with RKE2 tarball Airgap installation

rancher / rke2

https://docs.rke2.io/

Apache License 2.0

1.56k stars 268 forks source link

SELinux Trouble with RKE2 tarball Airgap installation #3381

Closed seb-835 closed 2 years ago

seb-835 commented 2 years ago

Environmental Info: RKE2 Version: 1.24.4+rke2r1 Node(s) CPU architecture, OS, and Version: 3.10.0-1160.71.1.el7.x86_64

Describe the bug: As soon as i set selinux in mode permissive, Only the kube-proxy restart, Other : etcd; api-server, scheduler, ... ...never goes up, so node is notReady.

Additional context / logs: the config.yaml have :

secret-encryption: true
selinux: true
profile: cis-1.6

Expected behavior: Node to be available in Selinux permissive with all kube components running.

I may have made a mistake, or forget settings, could you help ?

brandond commented 2 years ago

As soon as i set selinux in mode permissive, only the kube-proxy restart

What mode was selinux in when you initially started RKE2? Was selinux enabled in the rke2 configuration when you initially started it? Can you provide rke2-server and containerd logs showing why the pods aren't running?

seb-835 commented 2 years ago

Hi @brandond , i just redo an installation (in tarball airgap mode) on a fresh new server, with selinux set to permissive.

The containers does not pop up, in the logs of rke2-server i see :

SELinux is enabled for rke2 but process is not running in context 'container_runtime_t', rke2-selinux policy may need to be applied.

I attach rke2-server and containerd logs.

rke2-server.log containerd.log

Thanks for help, sure i may have missed to do something.

Martin-Weiss commented 2 years ago

@seb-835 - which OS are you using / RHEL7 or CentOS7? Maybe this is also related to https://github.com/rancher/rke2/issues/1865?

seb-835 commented 2 years ago

@Martin-Weiss RedHat Entreprise 7.9 look like the same issue, but i was not able to find any rke2 selinux policy file on the host.

Martin-Weiss commented 2 years ago

In case you do the RPM install instead of the tarball install - the installer also fetches the rke2-selinux policy RPM.. So if you can switch from tarball to RPM based install it should work on RHEL 7.9 - at least it has been working for me on CentOS 7.9. The problem seems to exist with the tarball install, only.

seb-835 commented 2 years ago

thanks @Martin-Weiss

you put me to the right way. i download the rpm rke2-selinux : rpm from https://github.com/rancher/rke2-selinux/releases i download the rpm : container-selinux , yum install both ...and got it works... so for tarball install , you need those 2 packages added to make it run in selinux mode.

Martin-Weiss commented 2 years ago

@seb-835 - so for your setup you are using the two RPMs rke2-selinux and container-selinux but the tarball of RKE2 1.24.4+rke2r1 and it works without any additional selinux policy change? Did you reboot after the deployment to see if all comes up well?

seb-835 commented 2 years ago

@Martin-Weiss yes, that's what i do, rke2 tarball + 2 rpm package : rke2-selinux and container-selinux no additional selinux policy add or change, and after rebooting some nodes to test : the cluster is still up with all its nodes running.

Martin-Weiss commented 2 years ago

@seb-835 - thanks - then there must be some other difference when comparing RHEL 7 to CentOS 7.9 with my issue https://github.com/rancher/rke2/issues/1865 ..

seb-835 commented 2 years ago

@Martin-Weiss Do you have cloud-provider enable in your env ?

Martin-Weiss commented 2 years ago

@Martin-Weiss Do you have cloud-provider enable in your env ?

I have been testing with Harvester - so yes.