containers / udica

This repository contains a tool for generating SELinux security profiles for containers
GNU General Public License v3.0
488 stars 47 forks source link

Possibly missing needed attributes for a given policy? #98

Closed perezjasonr closed 2 years ago

perezjasonr commented 3 years ago

Describe the bug

Udica could possibly be missing some attributes/rules for a given policy generation. I ran udica for a given container (in this case I was trying it for rook/ceph) that was getting avc denials, then updated the container to have the type, but it is still getting denied. Audit2allow still shows a potential rule that might have allowed the container to do what it was trying to do. explained below:

To Reproduce Steps to reproduce the behavior:

  1. attempt to install rook-ceph, in this case I am working on the init-container known as chown-container-data-dir for one of the deployments which is being denied (there are others but this is the first one being denied).

  2. find the crashing container, inspect and run udica against it. Here is the resulting policy generated:

cat chown-container-data-dir.cil
(block chown-container-data-dir
    (blockinherit container)
    (allow process var_lib_t ( dir ( add_name create getattr ioctl lock open read remove_name rmdir search setattr write )))
    (allow process var_lib_t ( file ( append create getattr ioctl lock map open read rename setattr unlink write )))
    (allow process var_lib_t ( sock_file ( append getattr open read write )))
    (allow process var_lib_t ( dir ( add_name create getattr ioctl lock open read remove_name rmdir search setattr write )))
    (allow process var_lib_t ( file ( append create getattr ioctl lock map open read rename setattr unlink write )))
    (allow process var_lib_t ( sock_file ( append getattr open read write )))
    (allow process var_lib_t ( dir ( add_name create getattr ioctl lock open read remove_name rmdir search setattr write )))
    (allow process var_lib_t ( file ( append create getattr ioctl lock map open read rename setattr unlink write )))
    (allow process var_lib_t ( sock_file ( append getattr open read write )))
  1. follow the instructions provided by udica in my case: semodule -i chown-container-data-dir.cil /usr/share/udica/templates/base_container.cil then update the container to use the new type/label.

  2. even with the new policy in place, the container is still being denied:

    type=SYSCALL msg=audit(1627570460.894:2008): arch=c000003e syscall=260 success=no exit=-13 a0=ffffff9c a1=55ef2249e3a0 a2=a7 a3=a7 items=0 ppid=77030 pid=84832 auid=4294967295 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=4294967295 comm="chown" exe="/usr/bin/coreutils" subj=system_u:system_r:chown-container-data-dir.process:s0:c82,c419 key=(null)
    type=AVC msg=audit(1627570460.894:2008): avc:  denied  { setattr } for  pid=84832 comm="chown" name="data" dev="nvme0n1p2" ino=33783456 scontext=system_u:system_r:chown-container-data-dir.process:s0:c82,c419 tcontext=system_u:object_r:container_var_lib_t:s0 tclass=dir permissive=0

notice the scontext: scontext=system_u:system_r:chown-container-data-dir.process:s0:c82,c419

Expected behavior the container with the policy in place, and with the type set to proper type it should get past the denial. this of course would mean (according to my understanding) it would not miss any needed allow rules (see below)

Additional context

These outputs might be useful lets see that specific source and target with class dir:

sesearch -A -s chown-container-data-dir.process -t container_var_lib_t -c dir
allow container_domain container_var_lib_t:dir { add_name getattr ioctl lock open read remove_name search write };
allow container_domain file_type:dir { getattr open search };
allow svirt_sandbox_domain container_var_lib_t:dir { add_name getattr ioctl lock open read remove_name search write };
allow svirt_sandbox_domain file_type:dir { getattr open search };

and audit2allow "thinks" an extra setattr is needed, only care about the first section specific to chown-container-data-dir.process, as I don't want to give entire container_t setattr obviously:

audit2allow -a
#============= chown-container-data-dir.process ==============
allow chown-container-data-dir.process container_var_lib_t:dir setattr;
#============= container_t ==============
#!!!! This avc can be allowed using the boolean 'container_manage_cgroup'
allow container_t cgroup_t:file write;
allow container_t container_var_lib_t:dir setattr;

possibly some significant sections from the inspect json:

crictl -r unix:///run/containerd/containerd.sock inspect a5c7874801937 | grep mountLab
el
        "mountLabel": "system_u:object_r:container_file_t:s0:c203,c480"

[root@ip-10-42-32-235 ~]# crictl -r unix:///run/containerd/containerd.sock inspect a5c7874801937 | grep selinux
        "selinuxRelabel": true
        "selinuxRelabel": true
        "selinuxRelabel": false
        "selinuxRelabel": false
        "selinuxRelabel": false
        "selinuxRelabel": true
        "selinuxRelabel": true
        "selinuxRelabel": true
          "selinux_relabel": true
          "selinux_relabel": true
          "selinux_relabel": true
          "selinux_relabel": true
          "selinux_relabel": true
          "selinux_options": {
        "selinuxLabel": "system_u:system_r:chown-container-data-dir.process:s0:c203,c480"

[root@ip-10-42-32-235 ~]# crictl -r unix:///run/containerd/containerd.sock inspect a5c7874801937 | grep selinux_options -C 3
          "namespace_options": {
            "pid": 1
          },
          "selinux_options": {
            "type": "chown-container-data-dir.process"
          },
          "run_as_user": {},

let me know if anything else might be useful, like the full inspect json.

Or if I missed something obvious...I apologize but thank you for your consideration.

perezjasonr commented 3 years ago

Ok so...this is probably a crucial update. but I learned about the append arg that udica has, after the append, it seems to have gotten past this. It looked like this:

ausearch -m avc -i > avcfile

udica --append-rules avcfile -j chown-container2.json chown-container-data-dir

policy now looks like this:

cat chown-container-data-dir.cil
(block chown-container-data-dir
    (blockinherit container)
    (allow process var_lib_t ( dir ( add_name create getattr ioctl lock open read remove_name rmdir search setattr write )))
    (allow process var_lib_t ( file ( append create getattr ioctl lock map open read rename setattr unlink write )))
    (allow process var_lib_t ( sock_file ( append getattr open read write )))
    (allow process var_lib_t ( dir ( add_name create getattr ioctl lock open read remove_name rmdir search setattr write )))
    (allow process var_lib_t ( file ( append create getattr ioctl lock map open read rename setattr unlink write )))
    (allow process var_lib_t ( sock_file ( append getattr open read write )))
    (allow process var_lib_t ( dir ( add_name create getattr ioctl lock open read remove_name rmdir search setattr write )))
    (allow process var_lib_t ( file ( append create getattr ioctl lock map open read rename setattr unlink write )))
    (allow process var_lib_t ( sock_file ( append getattr open read write )))
    (allow process container_var_lib_t ( dir ( setattr )))

avc denials have now moved on to a new denial and is back to container_t source (and the logs of the first container claim to have been successful).

If is all expected behavior, meaning, initial udica run may not catch everything, and a possible append using avc is standard procedure then I suppose this can be closed?

For now I will continue with these steps for the next denied container.

vmojzis commented 3 years ago

Udica doesn't always know enough about the container from crictl inspect, but this looks more like a misslabeled directory, or container runtime issue. Based on the AVC it seems that coreutils was trying to run chown on some directory named data labeled container_var_lib_t. It might be useful to know what was the exact path of that directory (you can gather it by rerunning the container with the original policy with full auditing enabled). This bugzilla looks very similar (container_t falls under container_domain as well), but is quite old. Also, could you please share the whole output of crictl inspect?

perezjasonr commented 3 years ago

but this looks more like a misslabeled directory, or container runtime issue.

may I know why you came to that conclusion? just for my own understanding? I just figured it was because the inspect as you said can't be enough sometimes.

And sure thing, based on out put the exact path for this particular container seems to be:

failed to change ownership of '/var/log/ceph' from root:root to ceph:ceph
failed to change ownership of '/var/lib/ceph/crash' from root:root to ceph:ceph
failed to change ownership of '/var/lib/ceph/mon/ceph-a' from root:root to ceph:ceph
chown: changing ownership of '/var/log/ceph': Permission denied
chown: changing ownership of '/var/lib/ceph/crash': Permission denied
chown: changing ownership of '/var/lib/ceph/mon/ceph-a': Permission denied

I can get another environment up for the full inspect output later today.

perezjasonr commented 3 years ago

heres the full inspect of that particular container:

{
  "status": {
    "id": "55c277d3973219bec55e8fc433ddde4612a92bcee898fbe15cc8a8264738ae9b",
    "metadata": {
      "attempt": 3,
      "name": "chown-container-data-dir"
    },
    "state": "CONTAINER_EXITED",
    "createdAt": "2021-08-02T15:01:03.864352829Z",
    "startedAt": "2021-08-02T15:01:04.027830706Z",
    "finishedAt": "2021-08-02T15:01:04.031468749Z",
    "exitCode": 1,
    "image": {
      "image": "docker.io/ceph/ceph:v15.2.8"
    },
    "imageRef": "docker.io/ceph/ceph@sha256:37939a3739e4e037dcf1b1f5828058d721d8c6de958212609f9e7d920b9c62bf",
    "reason": "Error",
    "message": "",
    "labels": {
      "io.kubernetes.container.name": "chown-container-data-dir",
      "io.kubernetes.pod.name": "rook-ceph-mon-a-57b6ffdd7d-gjd8c",
      "io.kubernetes.pod.namespace": "rook-ceph",
      "io.kubernetes.pod.uid": "f7abee70-4951-439a-bc6a-766dd766b3fb"
    },
    "annotations": {
      "io.kubernetes.container.hash": "fd9d89aa",
      "io.kubernetes.container.restartCount": "3",
      "io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
      "io.kubernetes.container.terminationMessagePolicy": "File",
      "io.kubernetes.pod.terminationGracePeriod": "30"
    },
    "mounts": [
      {
        "containerPath": "/etc/ceph",
        "hostPath": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~projected/rook-config-override",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": true,
        "selinuxRelabel": true
      },
      {
        "containerPath": "/etc/ceph/keyring-store/",
        "hostPath": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~secret/rook-ceph-mons-keyring",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": true,
        "selinuxRelabel": true
      },
      {
        "containerPath": "/var/log/ceph",
        "hostPath": "/var/lib/rook/rook-ceph/log",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": false,
        "selinuxRelabel": false
      },
      {
        "containerPath": "/var/lib/ceph/crash",
        "hostPath": "/var/lib/rook/rook-ceph/crash",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": false,
        "selinuxRelabel": false
      },
      {
        "containerPath": "/var/lib/ceph/mon/ceph-a",
        "hostPath": "/var/lib/rook/mon-a/data",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": false,
        "selinuxRelabel": false
      },
      {
        "containerPath": "/var/run/secrets/kubernetes.io/serviceaccount",
        "hostPath": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~secret/default-token-9xddr",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": true,
        "selinuxRelabel": true
      },
      {
        "containerPath": "/etc/hosts",
        "hostPath": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/etc-hosts",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": false,
        "selinuxRelabel": true
      },
      {
        "containerPath": "/dev/termination-log",
        "hostPath": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/containers/chown-container-data-dir/90a91e62",
        "propagation": "PROPAGATION_PRIVATE",
        "readonly": false,
        "selinuxRelabel": true
      }
    ],
    "logPath": "/var/log/pods/rook-ceph_rook-ceph-mon-a-57b6ffdd7d-gjd8c_f7abee70-4951-439a-bc6a-766dd766b3fb/chown-container-data-dir/3.log"
  },
  "info": {
    "sandboxID": "67c1906c2eb954298dfcb11b481386fc5075aa6e25a9ea8920280c9adf03af1c",
    "pid": 0,
    "removing": false,
    "snapshotKey": "55c277d3973219bec55e8fc433ddde4612a92bcee898fbe15cc8a8264738ae9b",
    "snapshotter": "overlayfs",
    "runtimeType": "io.containerd.runc.v2",
    "runtimeOptions": {
      "systemd_cgroup": true
    },
    "config": {
      "metadata": {
        "name": "chown-container-data-dir",
        "attempt": 3
      },
      "image": {
        "image": "sha256:5553b0cb212ca2aa220d33ba39d9c602c8412ce6c5febc57ef9cdc9c5844b185"
      },
      "command": [
        "chown"
      ],
      "args": [
        "--verbose",
        "--recursive",
        "ceph:ceph",
        "/var/log/ceph",
        "/var/lib/ceph/crash",
        "/var/lib/ceph/mon/ceph-a"
      ],
      "envs": [
        {
          "key": "CSI_RBDPLUGIN_METRICS_SERVICE_PORT_CSI_GRPC_METRICS",
          "value": "8081"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP_ADDR",
          "value": "10.103.252.173"
        },
        {
          "key": "KUBERNETES_SERVICE_HOST",
          "value": "10.96.0.1"
        },
        {
          "key": "KUBERNETES_SERVICE_PORT_HTTPS",
          "value": "443"
        },
        {
          "key": "ROOK_CEPH_MON_A_SERVICE_PORT",
          "value": "6789"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_3300_TCP_ADDR",
          "value": "10.106.49.41"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_SERVICE_HOST",
          "value": "10.108.229.146"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT",
          "value": "tcp://10.108.229.146:8080"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP",
          "value": "tcp://10.108.229.146:8080"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_SERVICE_PORT_CSI_GRPC_METRICS",
          "value": "8081"
        },
        {
          "key": "KUBERNETES_PORT_443_TCP_ADDR",
          "value": "10.96.0.1"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_6789_TCP",
          "value": "tcp://10.106.49.41:6789"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_SERVICE_PORT",
          "value": "8080"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT",
          "value": "tcp://10.103.252.173:8080"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP_PORT",
          "value": "8081"
        },
        {
          "key": "KUBERNETES_PORT_443_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "KUBERNETES_PORT_443_TCP_PORT",
          "value": "443"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP",
          "value": "tcp://10.103.252.173:8080"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP_ADDR",
          "value": "10.103.252.173"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP_PORT",
          "value": "8081"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP_PORT",
          "value": "8080"
        },
        {
          "key": "ROOK_CEPH_MON_A_SERVICE_HOST",
          "value": "10.106.49.41"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_6789_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_3300_TCP",
          "value": "tcp://10.106.49.41:3300"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_6789_TCP_ADDR",
          "value": "10.106.49.41"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_3300_TCP_PORT",
          "value": "3300"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP",
          "value": "tcp://10.103.252.173:8081"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP_ADDR",
          "value": "10.108.229.146"
        },
        {
          "key": "KUBERNETES_SERVICE_PORT",
          "value": "443"
        },
        {
          "key": "KUBERNETES_PORT",
          "value": "tcp://10.96.0.1:443"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT",
          "value": "tcp://10.106.49.41:6789"
        },
        {
          "key": "ROOK_CEPH_MON_A_SERVICE_PORT_TCP_MSGR2",
          "value": "3300"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_6789_TCP_PORT",
          "value": "6789"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_SERVICE_PORT_CSI_HTTP_METRICS",
          "value": "8080"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_SERVICE_PORT",
          "value": "8080"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_SERVICE_PORT_CSI_HTTP_METRICS",
          "value": "8080"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP",
          "value": "tcp://10.108.229.146:8081"
        },
        {
          "key": "ROOK_CEPH_MON_A_SERVICE_PORT_TCP_MSGR1",
          "value": "6789"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP_PORT",
          "value": "8080"
        },
        {
          "key": "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP_ADDR",
          "value": "10.108.229.146"
        },
        {
          "key": "KUBERNETES_PORT_443_TCP",
          "value": "tcp://10.96.0.1:443"
        },
        {
          "key": "ROOK_CEPH_MON_A_PORT_3300_TCP_PROTO",
          "value": "tcp"
        },
        {
          "key": "CSI_RBDPLUGIN_METRICS_SERVICE_HOST",
          "value": "10.103.252.173"
        }
      ],
      "mounts": [
        {
          "container_path": "/etc/ceph",
          "host_path": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~projected/rook-config-override",
          "readonly": true,
          "selinux_relabel": true
        },
        {
          "container_path": "/etc/ceph/keyring-store/",
          "host_path": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~secret/rook-ceph-mons-keyring",
          "readonly": true,
          "selinux_relabel": true
        },
        {
          "container_path": "/var/log/ceph",
          "host_path": "/var/lib/rook/rook-ceph/log"
        },
        {
          "container_path": "/var/lib/ceph/crash",
          "host_path": "/var/lib/rook/rook-ceph/crash"
        },
        {
          "container_path": "/var/lib/ceph/mon/ceph-a",
          "host_path": "/var/lib/rook/mon-a/data"
        },
        {
          "container_path": "/var/run/secrets/kubernetes.io/serviceaccount",
          "host_path": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~secret/default-token-9xddr",
          "readonly": true,
          "selinux_relabel": true
        },
        {
          "container_path": "/etc/hosts",
          "host_path": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/etc-hosts",
          "selinux_relabel": true
        },
        {
          "container_path": "/dev/termination-log",
          "host_path": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/containers/chown-container-data-dir/90a91e62",
          "selinux_relabel": true
        }
      ],
      "labels": {
        "io.kubernetes.container.name": "chown-container-data-dir",
        "io.kubernetes.pod.name": "rook-ceph-mon-a-57b6ffdd7d-gjd8c",
        "io.kubernetes.pod.namespace": "rook-ceph",
        "io.kubernetes.pod.uid": "f7abee70-4951-439a-bc6a-766dd766b3fb"
      },
      "annotations": {
        "io.kubernetes.container.hash": "fd9d89aa",
        "io.kubernetes.container.restartCount": "3",
        "io.kubernetes.container.terminationMessagePath": "/dev/termination-log",
        "io.kubernetes.container.terminationMessagePolicy": "File",
        "io.kubernetes.pod.terminationGracePeriod": "30"
      },
      "log_path": "chown-container-data-dir/3.log",
      "linux": {
        "resources": {
          "cpu_period": 100000,
          "cpu_shares": 2,
          "oom_score_adj": 1000,
          "hugepage_limits": [
            {
              "page_size": "1GB"
            },
            {
              "page_size": "2MB"
            }
          ]
        },
        "security_context": {
          "namespace_options": {
            "pid": 1
          },
          "run_as_user": {},
          "masked_paths": [
            "/proc/acpi",
            "/proc/kcore",
            "/proc/keys",
            "/proc/latency_stats",
            "/proc/timer_list",
            "/proc/timer_stats",
            "/proc/sched_debug",
            "/proc/scsi",
            "/sys/firmware"
          ],
          "readonly_paths": [
            "/proc/asound",
            "/proc/bus",
            "/proc/fs",
            "/proc/irq",
            "/proc/sys",
            "/proc/sysrq-trigger"
          ]
        }
      }
    },
    "runtimeSpec": {
      "ociVersion": "1.0.2-dev",
      "process": {
        "user": {
          "uid": 0,
          "gid": 0
        },
        "args": [
          "chown",
          "--verbose",
          "--recursive",
          "ceph:ceph",
          "/var/log/ceph",
          "/var/lib/ceph/crash",
          "/var/lib/ceph/mon/ceph-a"
        ],
        "env": [
          "PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin",
          "HOSTNAME=rook-ceph-mon-a-57b6ffdd7d-gjd8c",
          "CEPH_VERSION=octopus",
          "CEPH_POINT_RELEASE=-15.2.8",
          "CEPH_DEVEL=false",
          "CEPH_REF=octopus",
          "OSD_FLAVOR=default",
          "CSI_RBDPLUGIN_METRICS_SERVICE_PORT_CSI_GRPC_METRICS=8081",
          "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP_ADDR=10.103.252.173",
          "KUBERNETES_SERVICE_HOST=10.96.0.1",
          "KUBERNETES_SERVICE_PORT_HTTPS=443",
          "ROOK_CEPH_MON_A_SERVICE_PORT=6789",
          "ROOK_CEPH_MON_A_PORT_3300_TCP_ADDR=10.106.49.41",
          "CSI_CEPHFSPLUGIN_METRICS_SERVICE_HOST=10.108.229.146",
          "CSI_CEPHFSPLUGIN_METRICS_PORT=tcp://10.108.229.146:8080",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP=tcp://10.108.229.146:8080",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP_PROTO=tcp",
          "CSI_CEPHFSPLUGIN_METRICS_SERVICE_PORT_CSI_GRPC_METRICS=8081",
          "KUBERNETES_PORT_443_TCP_ADDR=10.96.0.1",
          "ROOK_CEPH_MON_A_PORT_6789_TCP=tcp://10.106.49.41:6789",
          "CSI_RBDPLUGIN_METRICS_SERVICE_PORT=8080",
          "CSI_RBDPLUGIN_METRICS_PORT=tcp://10.103.252.173:8080",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP_PROTO=tcp",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP_PORT=8081",
          "KUBERNETES_PORT_443_TCP_PROTO=tcp",
          "KUBERNETES_PORT_443_TCP_PORT=443",
          "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP=tcp://10.103.252.173:8080",
          "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP_ADDR=10.103.252.173",
          "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP_PORT=8081",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP_PORT=8080",
          "ROOK_CEPH_MON_A_SERVICE_HOST=10.106.49.41",
          "ROOK_CEPH_MON_A_PORT_6789_TCP_PROTO=tcp",
          "ROOK_CEPH_MON_A_PORT_3300_TCP=tcp://10.106.49.41:3300",
          "ROOK_CEPH_MON_A_PORT_6789_TCP_ADDR=10.106.49.41",
          "ROOK_CEPH_MON_A_PORT_3300_TCP_PORT=3300",
          "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP=tcp://10.103.252.173:8081",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP_ADDR=10.108.229.146",
          "KUBERNETES_SERVICE_PORT=443",
          "KUBERNETES_PORT=tcp://10.96.0.1:443",
          "ROOK_CEPH_MON_A_PORT=tcp://10.106.49.41:6789",
          "ROOK_CEPH_MON_A_SERVICE_PORT_TCP_MSGR2=3300",
          "ROOK_CEPH_MON_A_PORT_6789_TCP_PORT=6789",
          "CSI_RBDPLUGIN_METRICS_SERVICE_PORT_CSI_HTTP_METRICS=8080",
          "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP_PROTO=tcp",
          "CSI_CEPHFSPLUGIN_METRICS_SERVICE_PORT=8080",
          "CSI_CEPHFSPLUGIN_METRICS_SERVICE_PORT_CSI_HTTP_METRICS=8080",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8081_TCP=tcp://10.108.229.146:8081",
          "ROOK_CEPH_MON_A_SERVICE_PORT_TCP_MSGR1=6789",
          "CSI_RBDPLUGIN_METRICS_PORT_8081_TCP_PROTO=tcp",
          "CSI_RBDPLUGIN_METRICS_PORT_8080_TCP_PORT=8080",
          "CSI_CEPHFSPLUGIN_METRICS_PORT_8080_TCP_ADDR=10.108.229.146",
          "KUBERNETES_PORT_443_TCP=tcp://10.96.0.1:443",
          "ROOK_CEPH_MON_A_PORT_3300_TCP_PROTO=tcp",
          "CSI_RBDPLUGIN_METRICS_SERVICE_HOST=10.103.252.173"
        ],
        "cwd": "/",
        "capabilities": {
          "bounding": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FSETID",
            "CAP_FOWNER",
            "CAP_MKNOD",
            "CAP_NET_RAW",
            "CAP_SETGID",
            "CAP_SETUID",
            "CAP_SETFCAP",
            "CAP_SETPCAP",
            "CAP_NET_BIND_SERVICE",
            "CAP_SYS_CHROOT",
            "CAP_KILL",
            "CAP_AUDIT_WRITE"
          ],
          "effective": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FSETID",
            "CAP_FOWNER",
            "CAP_MKNOD",
            "CAP_NET_RAW",
            "CAP_SETGID",
            "CAP_SETUID",
            "CAP_SETFCAP",
            "CAP_SETPCAP",
            "CAP_NET_BIND_SERVICE",
            "CAP_SYS_CHROOT",
            "CAP_KILL",
            "CAP_AUDIT_WRITE"
          ],
          "inheritable": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FSETID",
            "CAP_FOWNER",
            "CAP_MKNOD",
            "CAP_NET_RAW",
            "CAP_SETGID",
            "CAP_SETUID",
            "CAP_SETFCAP",
            "CAP_SETPCAP",
            "CAP_NET_BIND_SERVICE",
            "CAP_SYS_CHROOT",
            "CAP_KILL",
            "CAP_AUDIT_WRITE"
          ],
          "permitted": [
            "CAP_CHOWN",
            "CAP_DAC_OVERRIDE",
            "CAP_FSETID",
            "CAP_FOWNER",
            "CAP_MKNOD",
            "CAP_NET_RAW",
            "CAP_SETGID",
            "CAP_SETUID",
            "CAP_SETFCAP",
            "CAP_SETPCAP",
            "CAP_NET_BIND_SERVICE",
            "CAP_SYS_CHROOT",
            "CAP_KILL",
            "CAP_AUDIT_WRITE"
          ]
        },
        "oomScoreAdj": 1000,
        "selinuxLabel": "system_u:system_r:container_t:s0:c255,c656"
      },
      "root": {
        "path": "rootfs"
      },
      "mounts": [
        {
          "destination": "/proc",
          "type": "proc",
          "source": "proc",
          "options": [
            "nosuid",
            "noexec",
            "nodev"
          ]
        },
        {
          "destination": "/dev",
          "type": "tmpfs",
          "source": "tmpfs",
          "options": [
            "nosuid",
            "strictatime",
            "mode=755",
            "size=65536k"
          ]
        },
        {
          "destination": "/dev/pts",
          "type": "devpts",
          "source": "devpts",
          "options": [
            "nosuid",
            "noexec",
            "newinstance",
            "ptmxmode=0666",
            "mode=0620",
            "gid=5"
          ]
        },
        {
          "destination": "/dev/mqueue",
          "type": "mqueue",
          "source": "mqueue",
          "options": [
            "nosuid",
            "noexec",
            "nodev"
          ]
        },
        {
          "destination": "/sys",
          "type": "sysfs",
          "source": "sysfs",
          "options": [
            "nosuid",
            "noexec",
            "nodev",
            "ro"
          ]
        },
        {
          "destination": "/sys/fs/cgroup",
          "type": "cgroup",
          "source": "cgroup",
          "options": [
            "nosuid",
            "noexec",
            "nodev",
            "relatime",
            "ro"
          ]
        },
        {
          "destination": "/etc/ceph",
          "type": "bind",
          "source": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~projected/rook-config-override",
          "options": [
            "rbind",
            "rprivate",
            "ro"
          ]
        },
        {
          "destination": "/dev/termination-log",
          "type": "bind",
          "source": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/containers/chown-container-data-dir/90a91e62",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/etc/hostname",
          "type": "bind",
          "source": "/var/lib/containerd/io.containerd.grpc.v1.cri/sandboxes/67c1906c2eb954298dfcb11b481386fc5075aa6e25a9ea8920280c9adf03af1c/hostname",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/etc/resolv.conf",
          "type": "bind",
          "source": "/var/lib/containerd/io.containerd.grpc.v1.cri/sandboxes/67c1906c2eb954298dfcb11b481386fc5075aa6e25a9ea8920280c9adf03af1c/resolv.conf",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/dev/shm",
          "type": "bind",
          "source": "/run/containerd/io.containerd.grpc.v1.cri/sandboxes/67c1906c2eb954298dfcb11b481386fc5075aa6e25a9ea8920280c9adf03af1c/shm",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/etc/hosts",
          "type": "bind",
          "source": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/etc-hosts",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/etc/ceph/keyring-store/",
          "type": "bind",
          "source": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~secret/rook-ceph-mons-keyring",
          "options": [
            "rbind",
            "rprivate",
            "ro"
          ]
        },
        {
          "destination": "/var/log/ceph",
          "type": "bind",
          "source": "/var/lib/rook/rook-ceph/log",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/var/lib/ceph/crash",
          "type": "bind",
          "source": "/var/lib/rook/rook-ceph/crash",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        },
        {
          "destination": "/var/run/secrets/kubernetes.io/serviceaccount",
          "type": "bind",
          "source": "/var/lib/kubelet/pods/f7abee70-4951-439a-bc6a-766dd766b3fb/volumes/kubernetes.io~secret/default-token-9xddr",
          "options": [
            "rbind",
            "rprivate",
            "ro"
          ]
        },
        {
          "destination": "/var/lib/ceph/mon/ceph-a",
          "type": "bind",
          "source": "/var/lib/rook/mon-a/data",
          "options": [
            "rbind",
            "rprivate",
            "rw"
          ]
        }
      ],
      "annotations": {
        "io.kubernetes.cri.container-name": "chown-container-data-dir",
        "io.kubernetes.cri.container-type": "container",
        "io.kubernetes.cri.sandbox-id": "67c1906c2eb954298dfcb11b481386fc5075aa6e25a9ea8920280c9adf03af1c"
      },
      "linux": {
        "resources": {
          "devices": [
            {
              "allow": false,
              "access": "rwm"
            }
          ],
          "memory": {},
          "cpu": {
            "shares": 2,
            "period": 100000
          }
        },
        "cgroupsPath": "kubepods-besteffort-podf7abee70_4951_439a_bc6a_766dd766b3fb.slice:cri-containerd:55c277d3973219bec55e8fc433ddde4612a92bcee898fbe15cc8a8264738ae9b",
        "namespaces": [
          {
            "type": "pid"
          },
          {
            "type": "ipc",
            "path": "/proc/70800/ns/ipc"
          },
          {
            "type": "uts",
            "path": "/proc/70800/ns/uts"
          },
          {
            "type": "mount"
          },
          {
            "type": "network",
            "path": "/proc/70800/ns/net"
          }
        ],
        "maskedPaths": [
          "/proc/acpi",
          "/proc/kcore",
          "/proc/keys",
          "/proc/latency_stats",
          "/proc/timer_list",
          "/proc/timer_stats",
          "/proc/sched_debug",
          "/proc/scsi",
          "/sys/firmware"
        ],
        "readonlyPaths": [
          "/proc/asound",
          "/proc/bus",
          "/proc/fs",
          "/proc/irq",
          "/proc/sys",
          "/proc/sysrq-trigger"
        ],
        "mountLabel": "system_u:object_r:container_file_t:s0:c255,c656"
      }
    }
  }
}
perezjasonr commented 3 years ago

Also I presume you might be interested in this, since you mentioned possible mislabeling, based on the inspect output:

ls -laZ /var/lib/rook/*
/var/lib/rook/mon-a:
total 0
drwxr-xr-x. 3 root root system_u:object_r:container_var_lib_t:s0 18 Aug  2 14:58 .
drwxr-xr-x. 4 root root system_u:object_r:container_var_lib_t:s0 36 Aug  2 14:58 ..
drwxr-xr-x. 2 root root system_u:object_r:container_var_lib_t:s0  6 Aug  2 14:58 data

/var/lib/rook/rook-ceph:
total 0
drwxr-xr-x. 4 root root system_u:object_r:container_var_lib_t:s0 30 Aug  2 14:58 .
drwxr-xr-x. 4 root root system_u:object_r:container_var_lib_t:s0 36 Aug  2 14:58 ..
drwxr-xr-x. 2 root root system_u:object_r:container_var_lib_t:s0  6 Aug  2 14:58 crash
drwxr-xr-x. 2 root root system_u:object_r:container_var_lib_t:s0  6 Aug  2 14:58 log
vmojzis commented 2 years ago

I reviewed the process of creating policies for mounts again and there actually does seem to be an issue there. Udica is using the context that selinux would assign to each "hostPath" instead of the one it actually has (matchpathcon <path>instead of ls -Z <path>). That is why your policy module only adds access to var_lib_t and not container_var_lib_t (matchpathcon /var/lib/rook/mon-a/data gives us var_lib_t). Thank you for reporting the issue and sorry it took me so long to understand.