Tanzu-Solutions-Engineering / tkg-lab

Day in the life of a TKG platform team.
140 stars 75 forks source link

Oauth 2: Failed to refresh token #71

Closed GuillaumeMorini closed 2 years ago

GuillaumeMorini commented 4 years ago

Hi,

I have followed the lab guide and deployed successfully all the steps. But after few hours, I lose the connection to my kubernetes cluster with the following error:

Unable to connect to the server: failed to refresh token: oauth2: cannot fetch token: 500 Internal Server Error
Response: {"error":"server_error"}

I can get it back to normal after downloading a new kubeconf. Could you help me to troubleshoot or configure to extend this or to autorefresh the token ?

Thanks

jaimegag commented 4 years ago

The initial token generated by Dex expires after 3 hours (we set it to 180 minutes in the Dex ConfigMap). After that we try to leverage the refresh-token against the IdP (Okta in this case, unless you used something else).

I will try to test and simulate asap to confirm, but it could be that we didn't enable Refresh Token grant type on the Okta side if you followed our instructions. To try to fix that, in the Okta UI: Applications > (Select you App) > General Settings > Allowed grant types > Check the "Refresh Token" checkbox Hopefully that may fix it.

GuillaumeMorini commented 4 years ago

It is stil not working even after enabling the Refresh token in Okta. I have the following logs in dex:

{"level":"info","msg":"login successful: connector \"oidc\", username=\"Alana Smith\", preferred_username=\"\", email=\"alana@winterfell.live\", groups=[\"Everyone\" \"platform-team\"]","time":"2020-07-09T11:42:47Z"}
{"level":"info","msg":"login successful: connector \"oidc\", username=\"Concourse CI\", preferred_username=\"\", email=\"concourse@winterfell.live\", groups=[\"Everyone\" \"platform-team\"]","time":"2020-07-09T11:43:32Z"}
{"level":"info","msg":"keys expired, rotating","time":"2020-07-09T14:45:10Z"}
{"level":"info","msg":"keys rotated, next rotation: 2020-07-09 20:45:11.152452028 +0000 UTC","time":"2020-07-09T14:45:11Z"}
{"level":"error","msg":"failed to refresh identity: oidc: failed to get refresh token: oauth2: token expired and refresh token is not set","time":"2020-07-09T16:09:21Z"}
{"level":"error","msg":"failed to refresh identity: oidc: failed to get refresh token: oauth2: token expired and refresh token is not set","time":"2020-07-09T16:09:21Z"}
{"level":"error","msg":"failed to refresh identity: oidc: failed to get refresh token: oauth2: token expired and refresh token is not set","time":"2020-07-09T16:09:21Z"}
GuillaumeMorini commented 4 years ago

Hello @jaimegag ,

Did you had a chance to test it, because it is still not working and require me a refresh each time and when I integrate with Concourse or GitLab it is a real pain in the ... to rotate the certificates each time ?

Thanks for your help and the awesome work

Guillaume

jaimegag commented 4 years ago

Hi @GuillaumeMorini . I did test it further and got to the same situation you are at. Just reached out to the engineering team in charge of the TKG extensions to se iff they can help troubleshooting. I'm not convinced this is an Okta issue anymore. I'll keep you posted

rojinsafavi commented 3 years ago

Any updates?

srwaggon commented 3 years ago

I've also encountered this issue as described by previous participants and will be subscribing for updates. Similarly, ensuring that refresh tokens are enabled from the application in Okta did not resolve my issue.

zjarjoura-wish commented 3 years ago

did anyone ever figure this out?

jaimegag commented 3 years ago

Sorry for the silence in this issue. There are some changes in the authentication workflow that will make this issue not applicable anymore. We are working on some changes that should be ready in a few weeks time. I'll be able to validate & confirm better then. Thanks for your patience!

zjarjoura-wish commented 3 years ago

Ah I see, I thought maybe someone had found a fix for Dex since I think this an issue on the Dex side with how it talks to Okta. Thanks anyway!

jaimegag commented 2 years ago

Closing this issue as it's outdated and not aligned with current version of the Product that leverages Pinniped to issue the tokens that are used by the client. Let's open a new issue if we can reproduce the a similar problem with TKG 1.4. I haven't been able to do so myself