el8: keep on truckin

[dweomer]

• keep up with container-selinux moving past 2.164.1 on el8

Signed-off-by: Jacob Blain Christen jacob@rancher.com

[dweomer]

The only problem with this currently is that upgrades from:

• container-selinux < 2.164.2 + k3s-selinux <= 0.4-1 :arrow_right: container-selinux >= 2.164.2 + k3s-selinux > 0.4 fail to leave the system in a correctly working state. The dnf/yum output, invoked via Vagrant, looks like this:

  default: Running transaction
  default:   Preparing        :                                                        1/1
  default:   Upgrading        : selinux-policy-3.14.3-67.el8_4.2.noarch                1/8
  default:   Running scriptlet: selinux-policy-3.14.3-67.el8_4.2.noarch                1/8
  default:   Running scriptlet: selinux-policy-targeted-3.14.3-67.el8_4.2.noarch       2/8
  default:   Upgrading        : selinux-policy-targeted-3.14.3-67.el8_4.2.noarch       2/8
  default:   Running scriptlet: selinux-policy-targeted-3.14.3-67.el8_4.2.noarch       2/8
  default:   Running scriptlet: container-selinux-2:2.167.0-1.module_el8.4.0+942+d25   3/8
  default:   Upgrading        : container-selinux-2:2.167.0-1.module_el8.4.0+942+d25   3/8
  default:   Running scriptlet: container-selinux-2:2.167.0-1.module_el8.4.0+942+d25   3/8
  default: Problems processing filecon rules
  default: Failed post db handling
  default: /usr/sbin/semodule:  Failed!
  default: 
  default:   Running scriptlet: k3s-selinux-0.5-7.el8.noarch                           4/8
  default:   Upgrading        : k3s-selinux-0.5-7.el8.noarch                           4/8
  default:   Running scriptlet: k3s-selinux-0.5-7.el8.noarch                           4/8
  default:   Cleanup          : k3s-selinux-0.4-1.el8.noarch                           5/8
  default:   Running scriptlet: k3s-selinux-0.4-1.el8.noarch                           5/8
  default:   Cleanup          : container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a   6/8
  default:   Running scriptlet: container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a   6/8
  default:   Cleanup          : selinux-policy-targeted-3.14.3-54.el8.noarch           7/8
  default:   Running scriptlet: selinux-policy-targeted-3.14.3-54.el8.noarch           7/8
  default:   Cleanup          : selinux-policy-3.14.3-54.el8.noarch                    8/8
  default:   Running scriptlet: selinux-policy-3.14.3-54.el8.noarch                    8/8
  default:   Running scriptlet: container-selinux-2:2.167.0-1.module_el8.4.0+942+d25   8/8
  default:   Running scriptlet: k3s-selinux-0.5-7.el8.noarch                           8/8
  default:   Running scriptlet: selinux-policy-3.14.3-54.el8.noarch                    8/8
  default:   Verifying        : container-selinux-2:2.167.0-1.module_el8.4.0+942+d25   1/8
  default:   Verifying        : container-selinux-2:2.164.1-1.module_el8.4.0+886+c9a   2/8
  default:   Verifying        : selinux-policy-3.14.3-67.el8_4.2.noarch                3/8
  default:   Verifying        : selinux-policy-3.14.3-54.el8.noarch                    4/8
  default:   Verifying        : selinux-policy-targeted-3.14.3-67.el8_4.2.noarch       5/8
  default:   Verifying        : selinux-policy-targeted-3.14.3-54.el8.noarch           6/8
  default:   Verifying        : k3s-selinux-0.5-7.el8.noarch                           7/8
  default:   Verifying        : k3s-selinux-0.4-1.el8.noarch                           8/8
  default: 
  default: Upgraded:
  default:   container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch
  default:   k3s-selinux-0.5-7.el8.noarch
  default:   selinux-policy-3.14.3-67.el8_4.2.noarch
  default:   selinux-policy-targeted-3.14.3-67.el8_4.2.noarch
  default: 
  default: Complete!

Notice this specifically:

    default: Problems processing filecon rules
    default: Failed post db handling
    default: /usr/sbin/semodule:  Failed!

This is the result of the conflicting file-contexts from the old k3s-selinux interfering with the installation of the overlap from the new container-selinux. The resulting "wrongness" can be detected quite easily:

$ [vagrant@localhost ~]$ sudo semanage fcontext --list | grep k3s
/etc/systemd/system/k3s.*                          regular file       system_u:object_r:container_unit_file_t:s0 
/usr/local/lib/systemd/system/k3s.*                regular file       system_u:object_r:container_unit_file_t:s0 
[vagrant@localhost ~]$ 

This tells us that the new k3s-selinux installed just fine while the new container-selinux did not. This is, however, rather straight-forwardly addressed via a follow-up reinstall of container-selinux:

[vagrant@localhost ~]$ sudo dnf reinstall container-selinux
Last metadata expiration check: 1:19:48 ago on Thu 14 Oct 2021 08:54:09 PM UTC.
Dependencies resolved.
=================================================================================================================================================================================================================================
 Package                                              Architecture                              Version                                                                       Repository                                    Size
=================================================================================================================================================================================================================================
Reinstalling:
 container-selinux                                    noarch                                    2:2.167.0-1.module_el8.4.0+942+d25aada8                                       appstream                                     52 k

Transaction Summary
=================================================================================================================================================================================================================================

Total download size: 52 k
Installed size: 48 k
Is this ok [y/N]: y
Downloading Packages:
container-selinux-2.167.0-1.module_el8.4.0+942+d25aada8.noarch.rpm                                                                                                                               3.0 kB/s |  52 kB     00:17    
---------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
Total                                                                                                                                                                                            3.0 kB/s |  52 kB     00:17     
Running transaction check
Transaction check succeeded.
Running transaction test
Transaction test succeeded.
Running transaction
  Preparing        :                                                                                                                                                                                                         1/1 
  Running scriptlet: container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        1/2 
  Reinstalling     : container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        1/2 
  Running scriptlet: container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        1/2 
  Cleanup          : container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        2/2 
  Running scriptlet: container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        2/2 
  Verifying        : container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        1/2 
  Verifying        : container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                        2/2 

Reinstalled:
  container-selinux-2:2.167.0-1.module_el8.4.0+942+d25aada8.noarch                                                                                                                                                               

Complete!
[vagrant@localhost ~]$ sudo semanage fcontext --list | grep k3s
/etc/systemd/system/k3s.*                          regular file       system_u:object_r:container_unit_file_t:s0 
/usr/bin/k3s                                       regular file       system_u:object_r:container_runtime_exec_t:s0 
/usr/lib/systemd/system/k3s.*                      regular file       system_u:object_r:container_unit_file_t:s0 
/usr/local/bin/k3s                                 regular file       system_u:object_r:container_runtime_exec_t:s0 
/usr/local/lib/systemd/system/k3s.*                regular file       system_u:object_r:container_unit_file_t:s0 
/var/lib/rancher/k3s(/.*)?                         all files          system_u:object_r:container_var_lib_t:s0 
/var/lib/rancher/k3s/agent/containerd/[^/]*/sandboxes(/.*)? all files          system_u:object_r:container_ro_file_t:s0 
/var/lib/rancher/k3s/agent/containerd/[^/]*/snapshots directory          system_u:object_r:container_ro_file_t:s0 
/var/lib/rancher/k3s/agent/containerd/[^/]*/snapshots/[^/]* directory          system_u:object_r:container_ro_file_t:s0 
/var/lib/rancher/k3s/agent/containerd/[^/]*/snapshots/[^/]*/.* all files          <<None>>
/var/lib/rancher/k3s/data(/.*)?                    all files          system_u:object_r:container_runtime_exec_t:s0 
/var/lib/rancher/k3s/storage(/.*)?                 all files          system_u:object_r:container_file_t:s0 
/var/run/k3s(/.*)?                                 all files          system_u:object_r:container_var_run_t:s0 
/var/run/k3s/containerd/[^/]*/sandboxes/[^/]*/shm(/.*)? all files          system_u:object_r:container_runtime_tmpfs_t:s0 
[vagrant@localhost ~]$ 

/cc @Oats87

[dweomer]

So, the SLE part of the build is failing for mysterious reasons (https://drone-pr.k3s.io/k3s-io/k3s-selinux/23/3/2):

+ dapper -f Dockerfile.microos.dapper policy/microos/scripts/build -- 2 | Sending build context to Docker daemon 200.7kB 3 | Step 1/9 : ARG TUMBLEWEED=opensuse/tumbleweed 4 | Step 2/9 : FROM ${TUMBLEWEED} 5 | latest: Pulling from opensuse/tumbleweed 6 | e25828d04cc8: Pulling fs layer 7 | e25828d04cc8: Verifying Checksum 8 | e25828d04cc8: Download complete 9 | e25828d04cc8: Pull complete 10 | Digest: sha256:54a4c072303106f4bbd5a653618b0f386b709061abd36b38349eacbbe0a1efa5 11 | Status: Downloaded newer image for opensuse/tumbleweed:latest 12 | ---> 166f1a5b27f5 13 | Step 3/9 : RUN zypper install -y container-selinux git rpm-build selinux-policy-devel 14 | ---> Running in 438a94fbf614 15 | terminate called after throwing an instance of 'std::system_error' 16 | what(): Operation not permitted 17 | The command '/bin/sh -c zypper install -y container-selinux git rpm-build selinux-policy-devel' returned a non-zero code: 139 18 | time="2021-10-14T22:05:27Z" level=fatal msg="exit status 139"

"what" indeed

[dweomer]

locally:

+ make -f /usr/share/selinux/devel/Makefile k3s.pp
make: gawk: Operation not permitted
make: /bin/sh: Operation not permitted
make: /bin/sh: Operation not permitted
make: /bin/sh: Operation not permitted
make: *** [/usr/share/selinux/devel/include/Makefile:168: tmp/all_interfaces.conf] Error 127
FATA[0154] exit status 2                                
make: *** [Makefile:19: microos-build] Error 1

WHAT?!

[dweomer]

OMG guess what?! I think it's this: https://github.com/moby/moby/issues/42680 /headdesk

[dweomer]

So I put a lock on glibc to prevent it from getting upgraded and of course other packages are balking:

Problem: the to be installed rpm-build-4.16.1.3-3.2.x86_64 requires 'libc.so.6(GLIBC_2.34)(64bit)', but this requirement cannot be provided
Problem: the to be installed selinux-tools-3.2-3.2.x86_64 requires 'libc.so.6(GLIBC_2.34)(64bit)', but this requirement cannot be provided

Problem: the to be installed rpm-build-4.16.1.3-3.2.x86_64 requires 'libc.so.6(GLIBC_2.34)(64bit)', but this requirement cannot be provided
  not installable providers: glibc-2.34-2.1.x86_64[repo-oss]
 Solution 1: remove lock to allow installation of glibc-2.34-2.1.x86_64[repo-oss]
 Solution 2: do not install rpm-build-4.16.1.3-3.2.x86_64
 Solution 3: break rpm-build-4.16.1.3-3.2.x86_64 by ignoring some of its dependencies

Choose from above solutions by number or skip, retry or cancel [1/2/3/s/r/c/d/?] (c): c
The command '/bin/sh -c zypper install -y container-selinux git rpm-build selinux-policy-devel' returned a non-zero code: 4
FATA[0027] exit status 4                                
make: *** [Makefile:19: microos-build] Error 1

[zdzichu]

Hi, just wanted to mention this change is very important. k3s-selinux is not installable on Fedora currently. Fedora ships container-selinux-2.170.0.

[dweomer]

Hi, just wanted to mention this change is very important. k3s-selinux is not installable on Fedora currently. Fedora ships container-selinux-2.170.0.

@zdzichu it is currently somewhat less than straightforward to upgrade even with this fix: https://bugzilla.redhat.com/show_bug.cgi?id=2014645

[dweomer]

@zdzichu please try with https://github.com/k3s-io/k3s-selinux/releases/tag/v0.5.testing.1

You will likely need to dnf reinstall -y container-selinux after upgrading (see https://bugzilla.redhat.com/show_bug.cgi?id=2014645)

[zdzichu]

v0.5.testing.1 works for me on both Fedora 34 and Fedora 35. Thank you!