spiffe / helm-charts-hardened

Apache License 2.0
17 stars 31 forks source link

Permission denied for agent with recommendations enabled. #411

Closed massaox closed 1 month ago

massaox commented 1 month ago

I was attempting the deploy the "helm charts hardened" with the recommendations set to true and noticed this warning/errors in the logs:

{
  "error": "failed to write data file: open /run/spire/agent-data.json.tmp: permission denied",
  "level": "warning",
  "msg": "Could not store SVID",
  "subsystem_name": "manager",
  "time": "2024-08-06T07:41:00Z"
}
{
  "error": "failed to write data file: open /run/spire/agent-data.json.tmp: permission denied",
  "level": "error",
  "msg": "Could not store bundle",
  "subsystem_name": "manager",
  "time": "2024-08-06T07:41:00Z"
}

Upon inspecting the process in the node I noticed it is running under the user 1000:

chronos    77687  0.1  0.3 1288640 55848 ?       Ssl  07:40   0:03  \_ /opt/spire/bin/spire-agent run -config /opt/spire/conf/agent/agent.conf

I performed a diff between the agent DaemonSet with the recommendations disabled (left side) and enabled (right side), and I could see the chart set the user 1000, as confirmed in my previous output:

      securityContext:                            securityContext:
        {}                            |         fsGroup: 1000
                                  >         fsGroupChangePolicy: OnRootMismatch
                                  >         runAsGroup: 1000
                                  >         runAsUser: 1000

I noticed an initContainer is added to address the permissions for the user 1000:

                                  >         - name: fsgroupfix
                                  >           image: cgr.dev/chainguard/bash:latest@sha256:8c9e5c
                                  >           imagePullPolicy: Always
                                  >           command: ["bash", "-c"]
                                  >           args:
                                  >             - "chown -R 1000:1000 /run/spire/agent-sockets /t
                                  >           resources:
                                  >             {}
                                  >           volumeMounts:
                                  >             - name: spire-agent-socket-dir
                                  >               mountPath: /run/spire/agent-sockets
                                  >             - name: spire-agent-admin-socket-dir
                                  >               mountPath: /tmp/spire-agent/private
                                  >           securityContext:
                                  >             runAsUser: 0
                                  >             runAsGroup: 0

However that does not address the directory /run/spire/ which the agent is trying to store the bundle and SVID information.

Chart version used spire-0.21.0 App Version: 1.9.6

Custom values used:

global:
  spire:
    recommendations:
      enabled: true
    namespaces:
      create: true
kfox1111 commented 1 month ago

Thanks for the issue. It seems related to https://github.com/spiffe/helm-charts-hardened/issues/396 as well. Will try and discuss this and the other issue at the sync today.