DataDog / helm-charts

Helm charts for Datadog products
Apache License 2.0
349 stars 1.02k forks source link

Error installing latest helm chart (3.76.0) on GKE Autopilot #1584

Closed jkebinger closed 1 month ago

jkebinger commented 1 month ago

We're using terraform to deploy datadog via helm chart. It had recently noted this

On GKE Autopilot, only one "datadog" Helm chart release is allowed by Kubernetes namespace due to the following new constraints on the Agent DaemonSet:
delete
* The serviceAccountName must be "datadog-agent".
delete
* All ConfigMap names mounted must be hardcode.

Today, i moved from version 3.70.7 to 3.76.0 in our staging cluster and it seems like that warning wasn't messing around. There aren't any instructions on https://docs.datadoghq.com/containers/kubernetes/distributions/?tab=helm#autopilot about this requirement or how to set the service account name to use and what permissions/roles such a "datadog-agent" service account needs

Error: cannot patch "datadog-agent" with kind DaemonSet: admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints. Violations details: {"[denied by autogke-default-linux-capabilities]":["linux capability 'SYS_ADMIN' on container 'agent' not allowed; Autopilot only allows the capabilities: 'AUDIT_WRITE,CHOWN,DAC_OVERRIDE,FOWNER,FSETID,KILL,MKNOD,NET_BIND_SERVICE,NET_RAW,SETFCAP,SETGID,SETPCAP,SETUID,SYS_CHROOT,SYS_PTRACE'."],"[denied by autogke-no-host-port]":["container agent specifies host ports [8125 4317 4318], which are disallowed in Autopilot.","container trace-agent specifies host ports [8126], which are disallowed in Autopilot."],"[denied by autogke-no-write-mode-hostpath]":["hostPath volume pointerdir in container agent is accessed in write mode; disallowed in Autopilot.","hostPath volume runtimesocketdir used in container agent uses path /var/run/containerd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume procdir used in container agent uses path /proc which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume cgroups used in container agent uses path /sys/fs/cgroup which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume logdockercontainerpath used in container agent uses path /var/lib/docker/containers which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume host-containerd-dir used in container agent uses path /var/lib/containerd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume runtimesocketdir used in container trace-agent uses path /var/run/containerd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume runtimesocketdir used in container process-agent uses path /var/run/containerd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume cgroups used in container process-agent uses path /sys/fs/cgroup which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume passwd used in container process-agent uses path /etc/passwd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume procdir used in container process-agent uses path /proc which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume procdir used in container init-config uses path /proc which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume runtimesocketdir used in container init-config uses path /var/run/containerd which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/]."]} Requested by user: 'tfc-service-account@pfab-cloud-staging.iam.gserviceaccount.com', groups: 'system:authenticated'.
tbavelier commented 1 month ago

Hello @jkebinger , This issue is coming from the SYS_ADMIN capability requested when datadog.sbom.containerImage.uncompressedLayersSupport is set to true, which is the default in 3.76.0. You can use --set datadog.sbom.containerImage.uncompressedLayersSupport=false to disable it in the meantime, while we are working on fixing this condition.

jkebinger commented 1 month ago

Thank you, that worked great