search

graydenshand / eks_jupyterhub

Jupyterhub on EKS
2 stars 1 forks source link

autoscaling not working yet? #3

Open stebo85 opened 2 months ago

stebo85 commented 2 months ago

Thank you for providing this great starting point for a jupyterhub deployment on eks. I started with your setup, but autoscaling doesn't work yet for me. Is there anything I need to configure in addition to get this to work?

Thank you Steffen

graydenshand commented 2 months ago

Hey @stebo85, thanks for opening this issue!

When I had originally set this up, I thought all that was required was the managed autoscaling node group. However, there is some additional configuration required to get the eks cluster to manage the autoscaling group (and actually create more nodes as new users join).

I will update this example as time allows, but here are some references to get you started.

Most of what you need should be spelled out here: https://github.com/kubernetes/autoscaler/blob/master/cluster-autoscaler/cloudprovider/aws/README.md

This section of the z2jh docs also has some relevant info https://z2jh.jupyter.org/en/stable/administrator/optimization.html#explicit-memory-and-cpu-allocated-to-core-pods-containers. Note, I was unable to get the "user placeholders" feature working on my private implementation (here's a topic about that issue).