I'm attempting to deploy Temporal on a GKE Autopilot cluster using Helm.
Describe the bug
When deploying Temporal on a GKE Autopilot cluster using Helm, I encounter constraints violations due to the configuration that Autopilot does not allow. This includes issues with hostNetwork, hostPID, hostPath, and privileged containers.
The violations details provided by the Helm installation failure are as follows:
Enabling hostNetwork and hostPID are not allowed in Autopilot.
Container node-exporter specifies host ports [9100], which are disallowed in Autopilot.
Several hostPath volume configurations are used that are not allowed in Autopilot. Allowed path prefixes for hostPath volumes in Autopilot are [/var/log/].
Container configure-sysctl is privileged, which is not allowed in Autopilot.
Minimal Reproduction
Create a GKE Autopilot cluster.
Use the Helm command to install Temporal: helm install temporaltest . --timeout 1200s
Observe the mentioned constraint violations.
Environment/Versions
OS and processor: M2 Mac
Temporal Version: 0.28.0
GKE Autopilot
Additional context
Output:
W0912 12:39:11.601304 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated DaemonSet default/temporaltest-prometheus-node-exporter: defaulted unspecified resources for containers [node-exporter] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.877105 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-kube-state-metrics: defaulted unspecified resources for containers [kube-state-metrics] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.877124 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-admintools: defaulted unspecified resources for containers [admin-tools] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.911690 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-grafana: defaulted unspecified resources for containers [download-dashboards, grafana] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.911741 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-prometheus-server: defaulted unspecified resources for containers [prometheus-server-configmap-reload, prometheus-server] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.924644 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-prometheus-pushgateway: defaulted unspecified resources for containers [pushgateway] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.941656 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-web: defaulted unspecified resources for containers [temporal-web] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.949917 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-matching: defaulted unspecified resources for containers [check-cassandra-service, check-cassandra, check-cassandra-temporal-schema, check-cassandra-visibility-schema, check-elasticsearch-index, temporal-matching] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.954034 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-history: defaulted unspecified resources for containers [check-cassandra-service, check-cassandra, check-cassandra-temporal-schema, check-cassandra-visibility-schema, check-elasticsearch-index, temporal-history] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.980734 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-worker: defaulted unspecified resources for containers [check-cassandra-service, check-cassandra, check-cassandra-temporal-schema, check-cassandra-visibility-schema, check-elasticsearch-index, temporal-worker] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:11.980904 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated Deployment default/temporaltest-frontend: defaulted unspecified resources for containers [check-cassandra-service, check-cassandra, check-cassandra-temporal-schema, check-cassandra-visibility-schema, check-elasticsearch-index, temporal-frontend] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:12.269257 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated StatefulSet default/temporaltest-alertmanager: defaulted unspecified resources for containers [alertmanager] (see http://g.co/gke/autopilot-defaults)
W0912 12:39:12.269257 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated StatefulSet default/elasticsearch-master: defaulted unspecified resources for containers [configure-sysctl] (see http://g.co/gke/autopilot-defaults), and adjusted resources to meet requirements for containers [elasticsearch] (see http://g.co/gke/autopilot-resources)
W0912 12:39:12.281192 56930 warnings.go:70] autopilot-default-resources-mutator:Autopilot updated StatefulSet default/temporaltest-cassandra: defaulted unspecified resources for containers [temporaltest-cassandra] (see http://g.co/gke/autopilot-defaults)
Error: INSTALLATION FAILED: 2 errors occurred:
* admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints.
Violations details: {"[denied by autogke-disallow-hostnamespaces]":["enabling hostNetwork is not allowed in Autopilot.","enabling hostPID is not allowed in Autopilot."],"[denied by autogke-no-host-port]":["container node-exporter specifies host ports [9100], which are disallowed in Autopilot."],"[denied by autogke-no-write-mode-hostpath]":["hostPath volume proc used in container node-exporter uses path /proc which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume sys used in container node-exporter uses path /sys which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/].","hostPath volume root used in container node-exporter uses path / which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [/var/log/]."]}
Requested by user: 'luka@******.com', groups: 'system:authenticated'.
* admission webhook "warden-validating.common-webhooks.networking.gke.io" denied the request: GKE Warden rejected the request because it violates one or more constraints.
Violations details: {"[denied by autogke-disallow-privilege]":["container configure-sysctl is privileged; not allowed in Autopilot"]}
Requested by user: 'luka@******.com', groups: 'system:authenticated'.
These are related to prometheus, not Temporal. We will soon be removing Prometheus from our helm charts, but either way, this is not something we can fix.
What are you really trying to do?
I'm attempting to deploy Temporal on a GKE Autopilot cluster using Helm.
Describe the bug
When deploying Temporal on a GKE Autopilot cluster using Helm, I encounter constraints violations due to the configuration that Autopilot does not allow. This includes issues with hostNetwork, hostPID, hostPath, and privileged containers.
The violations details provided by the Helm installation failure are as follows:
Minimal Reproduction
helm install temporaltest . --timeout 1200s
Environment/Versions
Additional context
Output: