search

pixie-io / pixie

Instant Kubernetes-Native Application Observability
https://px.dev
Apache License 2.0
5.38k stars 416 forks source link

Support GKE AutoPilot clusters #278

Open swithinfoote opened 3 years ago

swithinfoote commented 3 years ago

Is your feature request related to a problem? Please describe. Deployment to GKE Aotopilot clusters is not currently supported.

Describe the solution you'd like It would be great if we could deploy to our Autopilot clusters

Describe alternatives you've considered We can run a standard GKE cluster which is working fine.

Additional context

Create an Autopilot cluster in GKE and attempt to deploy. Unfortunately this fails currently.

Output from px deploy command

px deploy --kubeconfig /Users/***/.kube/config
Pixie CLI

Running Cluster Checks:
 ✔    Kernel version > 4.14.0 
 ✔    Cluster type is supported 
 ✔    K8s version > 1.12.0 
 ✔    Kubectl > 1.10.0 is present 
 ✔    User can create namespace 
 ✔    Cluster type is in list of known supported types 
Installing version: 0.7.12
Generating YAMLs for Pixie
Deploying Pixie to the following cluster: ***-autopilot
Is the cluster correct? (y/n) [y] : 
Found 5 nodes
 ✔    Creating namespace 
 ✔    Deleting stale Pixie objects, if any 
 ✔    Deploying secrets and configmaps 
 ✔    Deploying dependencies: NATS 
 ✕    Deploying Cloud Connector  ERR: admission webhook "validation.gatekeeper.sh" denied the request: [denied by autogke-no-write-mode-hostpath] hostPath volume sys used in container app uses path /sys which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: ["/var/log/"]. Requesting user: <***> and groups: <["system:authenticated"]>
FATA[0153] Failed to deploy Vizier                       error="admission webhook \"validation.gatekeeper.sh\" denied the request: [denied by autogke-no-write-mode-hostpath] hostPath volume sys used in container app uses path /sys which is not allowed in Autopilot. Allowed path prefixes for hostPath volumes are: [\"/var/log/\"]. Requesting user: <***> and groups: <[\"system:authenticated\"]>"
oazizi000 commented 3 years ago

GKE Autopilot has restrictions that makes it currently incompatible with BPF:

https://cloud.google.com/kubernetes-engine/docs/concepts/autopilot-overview?_ga=2.142566970.-940892016.1612382046#host_options_restrictions

We need access to the host namespaces, and Autopilot does not currently allow that.

mastersingh24 commented 2 years ago

@oazizi000 @swithinfoote - feel free to ping me directly and we can chat about how we can enable this for Autopilot.

oazizi000 commented 2 years ago

@mastersingh24 That'd be awesome! Can you join the Pixie community slack? pixie-community.slack.com

mastersingh24 commented 2 years ago

@mastersingh24 That'd be awesome! Can you join the Pixie community slack? pixie-community.slack.com

Done

sourcec0de commented 3 months ago

Looks like they published eBPF support. Not sure if the limitations originally mentioned by @oazizi000 are still present. https://cloud.google.com/blog/products/containers-kubernetes/ip-masquerading-and-ebpf-are-now-in-gke-autopilot