Automate process of building api tokens

[kyprifog]

Looking for opinions: Would it make sense to have an option to automate the api token creation and store it in a k8s secret so that when people first run the helm chart they don't have to do something like this:

export API_TOKEN=$(openssl rand -hex 32)

helm upgrade --install dhub dask/daskhub --set jupyterhub.hub.cookieSecret=${API_TOKEN} --set jupyterhub.proxy.secretToken=${API_TOKEN} --set jupyterhub.proxy.service.type=LoadBalancer --set jupyterhub.hub.services.dask-gateway.apiToken=${API_TOKEN} --set dask-gateway.gateway.auth.jupyterhub.apiToken=${API_TOKEN} --set jupyterhub.proxy.service.type=LoadBalancer

This would be in a similar vein to the lets-encrypt/ssl option.

I'm also doing something like this to pull in the same tokens when I upgrade the chart:

export API_TOKEN=$(helm get values data-ops -o json | jq '.daskhub."dask-gateway".gateway.auth.jupyterhub.apiToken' | sed -e 's/^"//' -e 's/"$//')

helm upgrade dhub . --reset-values --set jupyterhub.hub.cookieSecret=${API_TOKEN} --set jupyterhub.proxy.secretToken=${API_TOKEN} --set jupyterhub.proxy.service.type=LoadBalancer --set jupyterhub.hub.services.dask-gateway.apiToken=${API_TOKEN} --set dask-gateway.gateway.auth.jupyterhub.apiToken=${API_TOKEN}

Since --reuse-values just for this purpose can sometimes be a little too strong or have unintended consequences. I dont know if there is a way to set certain chart parameters as automatically "reused" in helm.

[consideRatio]

Technique in helm3 about this described in this issue, and helm2 has reached end of life with no security updates to come - tomorrow i think.

https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/1017

[kyprifog]

Ah perfect, closing in favor of https://github.com/jupyterhub/zero-to-jupyterhub-k8s/issues/1017

[consideRatio]

No please don't! That is to track the jupyterhub helm chart, not these helm charts as well ;D

Oh, it was mostly the z2jh helm chart that needed this kind of things i see. And there is a complexity for daskhub as helm chart to help dask-gateway make use of the peer secret I think.

Well, this can be improved in z2jh when its used in a standalone manner, I doubt it z2jh can help daskhub be self-installable without issues as it need to connect the secrets of z2jh with dask-gateway

[consideRatio]

Z2JH automatic secret generation

In z2jh 0.12.0+ / current master proxy.secretToken / hub.cookieSecret and such are now automatically generated if not set before once of installing 0.12.0+

Automate z2jh <-> dask-gateway token creation

daskhub needs a token known both by z2jh / dask-gateway so they can communicate safely and now they require it to be passed using helm chart configuration. If we let daskhub autogenerate it using a helm3 trick, then it cannot be made available as a helm template value for z2jh/dask-gateway. So, those charts will need to consume it as an environment variable instead or similar, which is then mounted on z2jh / dask-gateway.

Alternative improvement to z2jh <-> dask-gateway token creation

Another option is to still manually generate the token and set it on the root helm chart configuration level, and then use import-values to z2jh / dask-gateway configuration. See https://helm.sh/docs/topics/charts/.

My conclusion

I think both these options adds too much complexity given the maintenance capacity available in daskhub / dask-gateway to motivate the change.

[kyprifog]

Maybe i'm missing something, but why can't there be a template to create a k8s secret and pull into the respective charts that way?

[consideRatio]

I don't fully understand your question concretely enough to address it so I'll address it more generally.

1. dask-gateway and z2jh respectively do what they do as a Helm charts based on their configuration (combination of passed --values etc.), and a very relevant limitation is that these cannot be influenced by any logic in helm templates.
2. Due to 1., the only way without custom logic within dask-gateway and z2jh, daskhub needs to convey information to both dask-gateway and z2jh but given that we want to avoid manual setting of their helm chart configuration, and don't plan on using import-values from a common daskhub value, we are forced to create logic within dask-gateway and z2jh to have a way to learn about the information daskhub wants to pass to them using some other mechanism than passing chart configuration values.
3. My conclusion was providing a mechanism in dask-gateway and z2jh to become aware of information that daskhub would like to pass was adding a bit too much complexity in general. But, if anyone comes up with a robust implementation that is sustainable to maintain for all three helm charts, I'm absolutely open to it!

[kyprifog]

Ok I think that I was suggesting (2) in your list here, specifically:

(1). Have a template in daskhub that generates the tokens and stores it in a kubernetes secret (2). In daskhub's values.yaml for dask-gateway and z2jh respectively pass in overrides for the environment variables

      jupyterhub:
      extraEnv:
        TOKEN: 
          valueFrom:
            secretKeyRef:
              name: ${TOKEN} 
              key: TOKEN
      dask-gateway:
      extraEnv:
        TOKEN: 
          valueFrom:
            secretKeyRef:
              name: ${TOKEN} 
              key: TOKEN

Why were you wanting to avoid setting on each of the helm chart configurations? You said you wanted to avoid manually setting it, what if it was automatic by pulling from the secret thats generated by a secret template in the daskhub helm chart? (maybe this is what you meant by manually, idk).

As you noted this would also be useful:

https://github.com/helm/charts/issues/5167#issuecomment-619137759

[consideRatio]

Why were you wanting to avoid setting on each of the helm chart configurations? You said you wanted to avoid manually setting it, what if it was automatic by pulling from the secret thats generated by a secret template in the daskhub helm chart?

If I understand your question correctly, I avoid it because there is no way that I know of that lets daskhub generate a password with logic within its helm templates and passing it onwards to dependency charts like other --values / --set are passed.

Instead, you are forced to pass it indirectly by finding another way to communicate the generated token to z2jh / dask-gateway.

---

dask-gateway can individually generate a persistent token, and z2jh can individually generate one. But, what we need is that daskhub provides the same token to both without forcing z2jh or dask-gateway to establish a peer dependency where one chart blindly assumes the other exist etc. I don't want to merge a PR into z2jh with such peer dependency to make this feature possible at least.

Note that z2jh has added some automation for generating secrets in https://github.com/jupyterhub/zero-to-jupyterhub-k8s/pull/1993. To summarize, the goal would be to do this in daskhub for apitoken, but the challenge is to propegate the information to z2jh / dask-gateway.

[kyprifog]

@consideRatio Wouldn't the process I describe above be sufficient for generating the tokens and passing from daskhub down into the jupyterhub and dask-gateway via means of default values.yaml be sufficient? This is a pattern I have used on several helm charts in the past to establish default overrides. Building a template that generates the secret and then pulling the token down using the secret key words in k8s seems like it should work.

Maybe it would be better if I just submit a commit to illustrate, there may be issue with this approach I'm just not grokking right now.

[consideRatio]

Yeah try it, i dont see how you generate something in templates of a chart to then use them as helm template values to other subcharts.

[consideRatio]

I'm closing this in favor of https://github.com/dask/dask-gateway/issues/473 where I hope to provide a way for daskhub to no longer require the user generate and specify a token at all.