skypilot-org / skypilot

SkyPilot: Run AI and batch jobs on any infra (Kubernetes or 12+ clouds). Get unified execution, cost savings, and high GPU availability via a simple interface.
https://skypilot.readthedocs.io
Apache License 2.0
6.82k stars 513 forks source link

[k8s] Support exec based auth kubeconfigs on controllers #4379

Open romilbhardwaj opened 5 days ago

romilbhardwaj commented 5 days ago

GKE and EKS rely on exec based auth in the kubeconfig to authenticate with the k8s API server. This is currently not supported on SkyPilot controllers:

https://github.com/skypilot-org/skypilot/blob/ed4329a724aa583dd02e325e824fd1e36ee32aaf/sky/provision/kubernetes/utils.py#L911-L915

The current suggested workaround is to create a kubeconfig that uses token based auth with a service account (generate_kubeconfig.sh).

This workaround introduces friction for users, and may not be be feasible in environments where users cannot create service accounts.

We should support exec based auth, maybe starting with supporting GKE and EKS. This would require installing relevant dependencies and copying over the cloud credentials.