"systemctl start openshift" on Fedora 24 cause a lot of avc denied with start failure

[alezzandro]

Hi there,

I've managed to get a working adb-utils scripts on Fedora 23 without any modification, just installing the rpm for centos7. Anyway after upgrading to F24 the "systemctl start openshift" command keeps failing while SELinux complains about a: avc: denied { transition } for pid=**** comm="sh" path="/usr/bin/docker" ... (I've just tried the latest 2.0.1 version and it fails too)

If I try to troubleshoot the issue SELinux says that the current policies' set already includes the wanted policy (this seems too strange):

[root@freddy ~]# grep denied /var/log/audit/audit.log | tail
type=AVC msg=audit(1471889977.496:1196): avc:  denied  { transition } for  pid=20273 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889978.498:1197): avc:  denied  { transition } for  pid=20275 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889979.500:1198): avc:  denied  { transition } for  pid=20278 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889980.503:1199): avc:  denied  { transition } for  pid=20292 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889981.504:1200): avc:  denied  { transition } for  pid=20294 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889982.507:1201): avc:  denied  { transition } for  pid=20296 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889983.508:1202): avc:  denied  { transition } for  pid=20298 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889984.510:1203): avc:  denied  { transition } for  pid=20301 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889985.512:1204): avc:  denied  { transition } for  pid=20303 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0
type=AVC msg=audit(1471889986.514:1205): avc:  denied  { transition } for  pid=20305 comm="sh" path="/usr/bin/docker" dev="dm-2" ino=1992965 scontext=system_u:system_r:initrc_t:s0 tcontext=system_u:system_r:docker_t:s0 tclass=process permissive=0

[root@freddy ~]# grep denied /var/log/audit/audit.log | tail | audit2allow 

#============= initrc_t ==============

**#!!!! This avc is allowed in the current policy**
allow initrc_t docker_t:process transition;

I've managed to get it working again by removing all the occurences of bash interpreter: /usr/bin/sh, probabibly new releases of SELinux complain on the usage of bash interpreter invoked directly instead that through the one specified on the first line of the scripts.

This is the original file:

[root@freddy ~]# cat /usr/lib/systemd/system/openshift.service.bak 
[Unit]
Description=Docker Application Container for OpenShift
Documentation=https://docs.openshift.org/
After=docker.service
Requires=docker.service
Conflicts=etcd.service kube-apiserver.service kube-controller-manager.service kube-scheduler.service kube-proxy.service

[Service]
TimeoutStartSec=0
Restart=always
EnvironmentFile=-/etc/sysconfig/openshift_option
ExecStartPre=-/usr/bin/docker stop openshift
ExecStartPre=-/usr/bin/docker rm openshift
ExecStart=/usr/bin/sh /opt/adb/openshift/openshift
ExecStartPost=/usr/bin/sh /opt/adb/openshift/openshift_provision
ExecStop=/usr/bin/sh -c /opt/adb/openshift/openshift_stop
KillMode=process
Restart=on-failure
RestartSec=30s

[Install]
WantedBy=multi-user.target

This the one I edited:

[root@freddy ~]# cat /usr/lib/systemd/system/openshift.service
[Unit]
Description=Docker Application Container for OpenShift
Documentation=https://docs.openshift.org/
After=docker.service
Requires=docker.service
Conflicts=etcd.service kube-apiserver.service kube-controller-manager.service kube-scheduler.service kube-proxy.service

[Service]
TimeoutStartSec=0
Restart=always
EnvironmentFile=-/etc/sysconfig/openshift_option
ExecStartPre=-/usr/bin/docker stop openshift
ExecStartPre=-/usr/bin/docker rm openshift
ExecStart=/opt/adb/openshift/openshift
ExecStartPost=/opt/adb/openshift/openshift_provision
ExecStop=/opt/adb/openshift/openshift_stop
KillMode=process
Restart=on-failure
RestartSec=30s

[Install]
WantedBy=multi-user.target

I don't know if the Fedora support is in scope for this project, anyway I think this issue may break the scripts in next RHEL releases, so if you agree I can create a pull request and submit the fix.

Thanks, Best Regards.

[rhatdan]

I wonder if a policy update fixed this issue, which is why the AVC's say they are allowed in current policy. I would have figured these rules were always allowed though. Strange.

[praveenkumar]

@alezzandro Thank you for trying it on fedora24, (even support for fedora is not in the scope for this project) but I think we can work it more to make it available for fedora channel.

@rhatdan So you mean that issue which @alezzandro faced should be already part of allowed rules?

[rhatdan]

Yes I believe so.

[alezzandro]

I've also tried it on a fresh Fedora 24 vagrant machine,

The issue is the same (look at file attached below) with a lot of 'AVC avc: denied { transition }' adb-utils.txt

[praveenkumar]

@alezzandro Thanks. @rhatdan would it be more appropriate to open a bug from SELiunx side as you said it should be part of allowed rules?

[rhatdan]

How do you have /usr mounted? Do you have it mounted with nosuid?

[alezzandro]

No /usr mounted:

[root@localhost ~]# mount
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime,seclabel)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,seclabel,size=240628k,nr_inodes=60157,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev,seclabel)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,seclabel,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,seclabel,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,seclabel,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime,seclabel)
cgroup on /sys/fs/cgroup/net_cls,net_prio type cgroup (rw,nosuid,nodev,noexec,relatime,net_cls,net_prio)
cgroup on /sys/fs/cgroup/perf_event type cgroup (rw,nosuid,nodev,noexec,relatime,perf_event)
cgroup on /sys/fs/cgroup/cpuset type cgroup (rw,nosuid,nodev,noexec,relatime,cpuset)
cgroup on /sys/fs/cgroup/hugetlb type cgroup (rw,nosuid,nodev,noexec,relatime,hugetlb)
cgroup on /sys/fs/cgroup/memory type cgroup (rw,nosuid,nodev,noexec,relatime,memory)
cgroup on /sys/fs/cgroup/cpu,cpuacct type cgroup (rw,nosuid,nodev,noexec,relatime,cpu,cpuacct)
cgroup on /sys/fs/cgroup/pids type cgroup (rw,nosuid,nodev,noexec,relatime,pids)
cgroup on /sys/fs/cgroup/freezer type cgroup (rw,nosuid,nodev,noexec,relatime,freezer)
cgroup on /sys/fs/cgroup/blkio type cgroup (rw,nosuid,nodev,noexec,relatime,blkio)
cgroup on /sys/fs/cgroup/devices type cgroup (rw,nosuid,nodev,noexec,relatime,devices)
configfs on /sys/kernel/config type configfs (rw,relatime)
/dev/vda1 on / type ext4 (rw,relatime,seclabel,data=ordered)
selinuxfs on /sys/fs/selinux type selinuxfs (rw,relatime)
systemd-1 on /proc/sys/fs/binfmt_misc type autofs (rw,relatime,fd=24,pgrp=1,timeout=0,minproto=5,maxproto=5,direct)
mqueue on /dev/mqueue type mqueue (rw,relatime,seclabel)
hugetlbfs on /dev/hugepages type hugetlbfs (rw,relatime,seclabel)
debugfs on /sys/kernel/debug type debugfs (rw,relatime,seclabel)
tmpfs on /run/user/1001 type tmpfs (rw,nosuid,nodev,relatime,seclabel,size=50052k,mode=700,uid=1001,gid=1001)

[praveenkumar]

@alezzandro Can you please create a PR on that so I will add it to adb-utils next release?