bottlerocket-os / bottlerocket

An operating system designed for hosting containers
https://bottlerocket.dev
Other
8.73k stars 514 forks source link

SELinux seems to be blocking spire csi driver installation. #3833

Open empath-nirvana opened 7 months ago

empath-nirvana commented 7 months ago

I followed the spire/istio quick start here and I'm seeing a lot of selinux denials on the node:

[ 9787.965707] audit: type=1400 audit(1710880774.925:558): avc: denied { search } for pid=33158 comm="sh" name="spire-agent-socket-dir" dev="nvme1n1p1" ino=288654 scontext=system_u:system_r:container_t:s0:c244,c337 tcontext=system_u:object_r:data_t:s0:c839,c970 tclass=dir permissive=0

Image I'm using:

bottlerocket-aws-k8s-1.26-x86_64-v1.19.2-29cc92cc

What I expected to happen:

I should be able to mount the csi driver socket as a volume.

What actually happened:

I get permission denied errors.

How to reproduce the problem:

https://istio.io/latest/docs/ops/integrations/spire/

Follow the quick start tutorial in the istio documentation here, the ingress gateway won't come up because of permissions problems.

Maybe this is just a user error sort of thing and I need to fix the security context or something that needs to be fixed in the spire agent, but there's a work around here:

https://github.com/spiffe/spiffe-csi/issues/54#issuecomment-1490620403

that supposedly fixes the issue in OpenShift but doesn't resolve it for bottlerocket. (i get permission denied when trying to relabel the file)

vyaghras commented 7 months ago

@empath-nirvana Thankyou for bringing this up. I will try to replicate this and get back to you.

empath-nirvana commented 7 months ago

I resolved this -- for anybody else trying to figure this out -- you need to add this init container to the spire csi driver/agent pod

initContainers:
        - name: set-context
          image: ubuntu:22.04
          command: ["chcon", "-Rt", "container_file_t", "spire-agent-socket/"]
          volumeMounts:
          - name: spire-agent-socket-dir
            mountPath: /spire-agent-socket
          securityContext:
            seLinuxOptions:
              role: system_r
              user: system_u
              type: "super_t"