search

Scalr / agent-helm

Helm chart to install "scalr-agent" for connecting self-hosted runners and VCS to Scalr TACO
https://scalr.github.io/agent-helm/
5 stars 5 forks source link

GKE Autopilot Helm #62

Open Pravin-Selvaranjan opened 4 months ago

Pravin-Selvaranjan commented 4 months ago

I am trying to deploy this chart on an Autopilot GKE cluster, which limits access to cluster nodes. I was able to modify and use a persistent volume instead of the host nodes volume but when I attempt a run, the pods created for the run seem to also try and use the host node volumes.

Error I get is as below

Cannot create container "atask-v0oelok6rb7slrkuj". HTTP error 400. Admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: gke warden rejected the request because it violates one or more constraints. violations details: {"[denied by autogke-no-write-mode-hostpath]":["hostpath volume working-dir in container atask-v0oelok6rb7slrkuj is accessed in write mode; disallowed in autopilot.","hostpath volume data-dir in container atask-v0oelok6rb7slrkuj is accessed in write mode; disallowed in autopilot.","hostpath volume temporary-dir in container atask-v0oelok6rb7slrkuj is accessed in write mode; disallowed in autopilot.","hostpath volume run-plugins-dir in container atask-v0oelok6rb7slrkuj is accessed in write mode; disallowed in autopilot.","hostpath volume workspace-plugins-dir used in container atask-v0oelok6rb7slrkuj uses path /home/kubernetes/flexvolume/agent-k8s/workspaces/ws-v0o49lsfrfa68ueb2/plugins which is not allowed in autopilot. allowed path prefixes for hostpath volumes are: [/var/log/].","hostpath volume registry-terraform-io used in container atask-v0oelok6rb7slrkuj uses path /home/kubernetes/flexvolume/agent-k8s/plugins/registry.terraform.io which is not allowed in autopilot. allowed path prefixes for hostpath volumes are: [/var/log/].","hostpath volume registry-opentofu-org used in container atask-v0oelok6rb7slrkuj uses path /home/kubernetes/flexvolume/agent-k8s/plugins/registry.opentofu.org which is not allowed in autopilot. allowed path prefixes for hostpath volumes are: [/var/log/].","hostpath volume global-plugins-dir-v012 used in container atask-v0oelok6rb7slrkuj uses path /home/kubernetes/flexvolume/agent-k8s/plugins/linuxAmd64 which is not allowed in autopilot. allowed path prefixes for hostpath volumes are: [/var/log/]."]} requested by user: 'system:serviceaccount:scalr:scalr-agent-agent-k8s', groups: 'system:serviceaccounts,system:serviceaccounts:scalr,system:authenticated'

mermoldy commented 4 months ago

@Pravin-Selvaranjan Hi, sorry, but currently the agent does not support GKE Autopilot mode because hostPath volumes are required for the agent to work. You can find some details here: https://github.com/Scalr/agent-helm/issues/5#issuecomment-1624445809. We want to get rid of hostPath volumes in the future and make agent fully cloud-native (and so compatible with GKE Autopilot), but there are no explicit roadmap at the moment.