headlamp-k8s / headlamp

A Kubernetes web UI that is fully-featured, user-friendly and extensible
https://headlamp.dev
Apache License 2.0
2.22k stars 156 forks source link

Issue with OIDC for websocket connections after upgrade from 0.18.0 to 0.19.0 #1324

Closed SergK closed 1 year ago

SergK commented 1 year ago

After Upgrading from Headlamp 0.18.0 to 0.19.0, my OIDC configuration stops working for WebSocket connections.

Headlamp 0.18.0 checking with websocat I have 403 - which is expected:

websocat 'wss://headlamp.example.com/clusters/main/apis/v2.edp.epam.com/v1/namespaces/edp/cdpipelines?watch=1&resourceVersion=1108244993'
websocat: WebSocketError: WebSocketError: Received unexpected status code (403 Forbidden)
websocat: error running

Headlamp 0.19.0 checking with websocat I have 502 - which is not expected:

websocat 'wss://headlamp.example.com/clusters/main/apis/v2.edp.epam.com/v1/namespaces/edp/cdpipelines?watch=1&resourceVersion=1108244993'
websocat: WebSocketError: WebSocketError: Received unexpected status code (502 Bad Gateway)
websocat: error running

And in the headlamp logs, I see:

2023/08/14 06:42:01 http: proxy error: No valid id-token, and cannot refresh without refresh-token
2023/08/14 06:42:01 http: proxy error: No valid id-token, and cannot refresh without refresh-token

here is the complete log from the container start

2023/08/14 05:51:56 static plugin dir: 
2023/08/14 05:51:56 plugins-dir: /headlamp/plugins
2023/08/14 05:51:56 dynamic clusters support: false
2023/08/14 05:51:56 Helm support: false
2023/08/14 05:51:56 Proxy URLs: []
{"level":"info","time":"2023-08-14T05:51:56Z","message":"Proxy setup for context \"main\" to cluster url \"https://10.100.0.1:443\""}
*** Headlamp Server ***
  API Routers:
2023/08/14 05:51:56 Error loading kubeconfig: read /: is a directory
2023/08/14 05:51:56 Error loading dynamic kubeconfig file: open /home/headlamp/.config/Headlamp/kubeconfigs/config: no such file or directory
2023/08/14 06:41:53 http: proxy error: No valid id-token, and cannot refresh without refresh-token
2023/08/14 06:41:55 http: proxy error: No valid id-token, and cannot refresh without refresh-token
2023/08/14 06:41:55 http: proxy error: No valid id-token, and cannot refresh without refresh-token

It seems that the error returns from here: https://github.com/headlamp-k8s/headlamp/blob/v0.19.0/backend/pkg/kubeconfig/kubeconfig.go#L113

yolossn commented 1 year ago

Hey @SergK Can you confirm if this is happening only for websockets?

SergK commented 1 year ago

Hi @yolossn, it's true that I can log in with OIDC, but I'm unable to receive "live updates" and have to refresh the page instead.

yolossn commented 1 year ago

Just an update on the progress. I am able to reproduce this issue and the websocket connection issue is happening only when using OIDC authentication. The No valid id-token, and cannot refresh without refresh-token response is from the Kubernetes API server and I don't see any difference in the request made from the frontend or the request that is forwarded to the Kubernetes API server by the backend.

yolossn commented 1 year ago

Hello @SergK, I've created a Docker image that includes the necessary fix and pushed it to docker.io/yolossn/headlamp-oidc-fix:latest. Could you please check to confirm if this resolves the problem?

SergK commented 1 year ago

Hello @yolossn, everything seems to be working perfectly without any issues.

yolossn commented 1 year ago

Hey, Thanks for the update. Ill try to get this released ASAP.

SergK commented 1 year ago

@yolossn Thank you for quick feedback