search

pangeo-data / helm-chart

Pangeo helm charts
https://pangeo-data.github.io/helm-chart/
21 stars 26 forks source link

Automate setup of dask-gateway jupyterhub service #126

Closed jhamman closed 3 years ago

jhamman commented 4 years ago

We currently have a line in our chart that sets the URL of the dask-gateway service on the hub:

https://github.com/pangeo-data/helm-chart/blob/6749ba1276f0eacc68bf23be4224504403cd2e99/pangeo/values.yaml#L39-L43

Unfortunately, this only works for our dev-staging deployment. How can we automatically determine this URL. Any Helm/k8s magic that can make this possible?

cc @jcrist

tjcrone commented 4 years ago

Is there a reason why we cannot just hard code it in each deployments configuration? I have done this for OOI, and it seems to be working. Still not able to access the dashboard but I just got it started.

jhamman commented 4 years ago

No reason except that it should be easily automated. For now, everyone should be overwriting this url value with their own deployment.

tjcrone commented 4 years ago

Yeah seems like it would be possible but I don't know what helm uses as the deployment variable off the top of my head.

On a related note, I have not seen any way to use a predefined static IP address for the two dask-gateway LoadBalancers, like we do for the proxy-public LoadBalancer. Have you seen where that configuration would live if it exists?

jcrist commented 4 years ago

On a related note, I have not seen any way to use a predefined static IP address for the two dask-gateway LoadBalancers, like we do for the proxy-public LoadBalancer. Have you seen where that configuration would live if it exists?

You'd set dask-gateway.schedulerProxy.service.loadBalancerIP and dask-gateway.webProxy.service.loadBalancerIP. See https://gateway.dask.org/install-kube.html#helm-chart-reference for the dask-gateway chart docs.

jcrist commented 4 years ago

How can we automatically determine this URL. Any Helm/k8s magic that can make this possible?

On initial investigation, it looks like helm doesn't allow templating values for subcharts (there's a few issues/stackoverflows about this, see e.g. https://stackoverflow.com/questions/55425688/helm-programmatically-override-subchart-values-yaml). My understanding is that if we wanted this behavior we'd need to use another tool to generate the values.yaml file for us. Yaml generating yaml generating yaml... I'm not sure what the best option is here.

tjcrone commented 4 years ago

This appears to be working for me: https://github.com/ooicloud/pangeo-cloud-federation/commit/432ec42490cb0eadb79fbb730b446e189ca2cea0. I will do some more testing and then PR this.

tjcrone commented 4 years ago

Nope my bad. That did not appear to work. @jcrist was correct. 😁

Since we have so few deployments and it is just one line, seems like it would be okay to just plug this line into everyone's common.yaml file.

@jcrist thank you for the tips on defining the IP address!

TomAugspurger commented 3 years ago

This is done properly in dask/daskhub.