headlamp-k8s / headlamp

A Kubernetes web UI that is fully-featured, user-friendly and extensible
https://headlamp.dev
Apache License 2.0
2.03k stars 148 forks source link

Allow Custom API-Server Endpoint #1192

Open Jaguar-Kwok opened 1 year ago

Jaguar-Kwok commented 1 year ago

For some use-case, especially on public cloud, some request have to pass through some proxy for handling. E.g. kube-oidc-server if OIDC issuer is from private endpoint. A flag and helm chart value can be also added for this item. https://aws.amazon.com/blogs/opensource/consistent-oidc-authentication-across-multiple-eks-clusters-using-kube-oidc-proxy/

illume commented 1 year ago

Hello @Jaguar-Kwok

could you please expand a little bit on what you need from Headlamp for this? To help my understanding.

It sounds like you would like:

We already reverse proxy api endpoints with the headlamp server, so I’m wondering if this feature might be best or easily added directly to Headlamp. Since we want to support managed Kubernetes providers better in Headlamp, I wonder if this will be a very common need?

Jaguar-Kwok commented 1 year ago

Hi @illume Thank you for your response and the detailed explanation. Your insights align perfectly with my understanding. As for the configurable API server endpoint, I believe one is enough for my use case and I am not sure if others have use case with multiple API server endpoint.

I am currently in the process of refining some ideas and code, but you may find a useful reference in https://github.com/headlamp-k8s/headlamp/commit/efa9af2ea906230ec78db24166c6562b628225b5.

I am confident that this feature will significantly enhance the functionality of Headlamp. Given that public Cloud providers often offer less customization for the managed API-server, this feature becomes helpful. E.g. AWS EKS does not allow private subnet OIDC service to be set in the oidc setting and I am using Dex inside my kubernetes cluster as the oidc auth provider.

Another way to approach this problem is to allow oidc login to clusters other than in-cluster cluster. Such that I can put my proxy just as another kubernetes api server and attatch an oidc auth method to that. But I am not sure if this allign the overall development timeline of headlamp. ctxtProxy, ok := c.contextProxies["test123"] if !ok { http.NotFound(w, r) } //fmt.Println("forward to kube-OIDC-proxy") handler := proxyHandler(server, ctxtProxy.proxy) handler(w, r) return Here is some bad code that I used to create a functional prototype in my testing cluster. It uses incluster proxy in order to auth with oidc but then further forward all the request to a cotent proxy that create from the kubeconfig file. So things should be compatable, but at the same time, it kind of violate current approach which only using kube config file to create a list of proxy, as definition of oidc auth provider. (which leads to even more problem such as multi-cluster one oidc authoizer and multi-cluster multi oidc authoizer )

Many thanks.

Jaguar-Kwok commented 1 year ago

Please allow me to suppliment with some diagram.

Headlamp Share-AWS drawio (1)

This is the architecture of one of my cluster. Due to the limitation of AWS EKS, the api server can only set public OIDC provider. However, due to privacy and security concern, we will use dex with connection to AD through LDAP. Hence the kube-oidc-proxy is needed or else the api server is unable to validate the OIDC token.

gecube commented 4 months ago

Good day, is there any success to run headlamp with kube-oidc-proxy? I use the same architecture like @Jaguar-Kwok