Continuously refreshing refresh-token and id-token

[lokotei]

Hi team! We're facing with some issue. Openunison refreshing id-token and refresh-token every ~60 seconds and updating local users ~/.kube/config Which is not comfortable when users are working with couple of k8s tools at the same time. It doesn't matter which tool's your/we will use, but each 60 second you will got a disconnect. Is there any possibility to disable/increase it?

[mlbiam]

Openunison refreshing id-token and refresh-token every ~60 seconds and updating local users ~/.kube/config

Yes, this is by design. id_tokens are bearer tokens, so anyone who has it can use it. If someone were to get the token because of a bug somewhere in the infrastructure an attacker could use that token against you. The 60 second timeout on the token means that once an attacker gets a token, it's unlikely they'll be able to use it.

It doesn't matter which tool's your/we will use, but each 60 second you will got a disconnect.

Can you elaborate on disconnected? Does the token not get refreshed? Can you share which tools are being used?

Is there any possibility to disable/increase it?

We can add it to the helm chart, but honestly I'd like to dig deeper into your use case and issues before going that route to see if there's a better approach.

[lokotei]

Thanks a lot @mlbiam for the quick reply.

Can you elaborate on disconnected? Does the token not get refreshed? Can you share which tools are being used?

Tokens are refreshing and thats the problem :)

So we're using kubectl, k9s, telepresence, lens etc. And some of them in parallel. At some point of time (i believe until the moment when new tokens are updated in ~/.kube/config) on of these tools got disconnection, because of not authorize (401) I believe also if you will follow the same steps you will got the same behavior :) In short:

• install for example lens
• setup a cluster/context
• execute for example next command watch -n1 kubectl get sa
• left it as it is
• go to standard therminal
• configure the same cluster/context
• execute the same command watch -n1 kubectl get sa
• wait a ~60s
• check both terminals

Of course after pressing "Reconnect" button you will connect again, because it will re-read the config file with the new tokens. But connection was interrupted....

I know, maybe that's not popular case, but for us it's important as we need to work with some long living connection. And i hope you will provide us some solution. Thanks a lot!

[mlbiam]

I see. So I think the best solution is to add a grace period for refresh tokens so that a lack of locking doesn't cause your various tools to break on refresh. But that's a feature we need to add and that would take a while. We'll get a configurable id_token time length into the helm charts as a short term solution so you can extend the life of your id_token and reduce refreshs

[lokotei]

Would be great if you can implement it. This would be much appreciated. Thanks a lot!

[mlbiam]

looks like its already there, just not documented. in your values.yaml add K8S_TOKEN_LIFE_MILLIS: "600000" to openunison.non_secret_data to make the time an hour and redeploy openunison using the ouctl command.

[lokotei]

Hi @mlbiam Sorry for the late reply. Thanks for the update. Maybe the last question from us, in case we'll need to extend this ttl to 24h will it work? Or the 10 mins is maximum?

[mlbiam]

24 hours should work. Haven't tested it though.

[lokotei]

Hello @mlbiam Unfortunately it doesn't work :( [2022-12-15 00:26:12,913][XNIO-1 task-1] WARN OpenIDConnectIdP - could not load trusts java.lang.NumberFormatException: For input string: "8.64e+07" at java.lang.NumberFormatException.forInputString(NumberFormatException.java:65) ~[?:?] It looks like it can't be set more than 1M ms Could you provide us some fix/workaround here? Thanks a lot for your help!

[mlbiam]

maybe go for smoething shorter? like 8 hours? I'm adding the refresh token grace period for our next release. hoping to get it done before the end of December.

[mlbiam]

@lokotei i'm not able to reproduce this issue. i've tried on macOS, Linux, and Windows with multiple kubectl watches and octant running at the same time. Everything refreshes properly. Can you share what some of the tools are that you're using?

[lokotei]

Hi @mlbiam as you have mention we're trying to use K8S_TOKEN_LIFE_MILLIS and it's in ms, right? Did you tried to use 20 minutes or more? We did try, and looks like the tool can align with this value bigger that 1M ms (~16,66 minutes) Or it can be set on the other way ? And it doesn't matter which OS or k8s tool you will use, that comes from openunison. If it's somehow works for you, could you please share somewhere your values.yaml with us? Just to compare.

[mlbiam]

I'm just using the regular config with a 1m timeout. Not having issues with multiple tools running at the same time. Can you share what tools you are using?

[lokotei]

Sorry, maybe my bad and wrong explanation, 1M ms = 1000000ms (~16,66 minutes) Right now i'm using kubectl/k9s/k8slens.dev, but even with single k9s we have this behavior. It works with the ttl < 1Mi ms but if we're trying to set more, than deployment failed with this error: [2022-12-15 00:26:12,913][XNIO-1 task-1] WARN OpenIDConnectIdP - could not load trusts java.lang.NumberFormatException: For input string: "8.64e+07" at

[mlbiam]

Sorry, I think we're talking about two different things. I'm trying to setup a test for refresh tokens having a grace period and am having issues reproducing having the refresh token fail.

[lokotei]

Hi @mlbiam ! Sorry for boring you. But is there any updates on our issue?

[mlbiam]

Yes, we've added two new features to account for what's a well known issue with the client-go sdk where refresh tokens aren't being managed well:

1. https://github.com/TremoloSecurity/OpenUnison/issues/719
2. https://github.com/TremoloSecurity/OpenUnison/issues/724

Between supporting a grace period for refresh tokens and making it so each load of the token screen gives you a fresh set of id_tokens and refresh_tokens you'll be able to either have different kubectl configuration files or let multiple tools work off of one file, but with some leeway to keep the tools from breaking the config. Here's a preview of the doc we wrote: https://github.com/OpenUnison/openunison.github.io/blob/1.0.32/docs/knowledgebase/sessions.md

We haven't cut a release of 1.0.33 (which includes these new features), but you can use a beta:

Get the beta repo:

helm repo add tremolo-betas https://nexus.tremolo.io/repository/helm-betas/
helm repo update

In your values.yaml, set your image to docker.io/tremolosecurity/betas:1.0.33. If you want to enable refresh_token grace periods, follow https://github.com/OpenUnison/openunison.github.io/blob/1.0.32/docs/knowledgebase/sessions.md#refresh-token-grace-period

Finally, upgrade using ouctl:

ouctl install-auth-portal --operator-chart=tremolo-betas/openunison-operator -c tremolo-betas/orchestra -l tremolo-betas/orchestra-login-portal /path/to/values.yaml

Once OpenUnison is running again, both the above features will be available.

[lokotei]

Hi @mlbiam Maybe the last question from us. When do you plan update the stable releases (e.g. helm-charts) with those changes?

[mlbiam]

We had to roll in a few other updates. Right now we are planning on the last week of January or first week of February. Here's the roadmap for the 1.0.33 release: https://github.com/TremoloSecurity/OpenUnison/milestone/30

[lokotei]

Thanks a lot @mlbiam !

[mlbiam]

@lokotei this is now available in the main OpenUnison build - https://openunison.github.io/knowledgebase/sessions/