snowex-hackweek / jupyterhub

jupyterhub configuration for snowex hackweek 2021
https://snowex.hackweek.io
MIT License
1 stars 0 forks source link

DOC: Per-user resources configuration #10

Open scottyhq opened 3 years ago

scottyhq commented 3 years ago

Just want to document where per-user resources are defined, which are a combination of cloud-provider and hub configuration settings:

for CPU and RAM, we have:

  1. cloud provider https://github.com/snowex-hackweek/jupyterhub/blob/c988de715ae7179e5ab2813af7dec4900224862d/terraform/eks/main.tf#L97-L99

  2. jupyterhub configuration https://github.com/snowex-hackweek/jupyterhub/blob/c988de715ae7179e5ab2813af7dec4900224862d/hub/config.yaml#L4-L9

So, for our initial configuration, the instance type has cVPU=8 and RAM=32GB, and each user is guaranteed 1vCPU and 7GB RAM. So in this case, the RAM request limits the number of users per node (32//7 = 4).

Storage allocations are less obvious because we're using defaults:

  1. The user home directory has a 10GB dedicated disk: https://zero-to-jupyterhub.readthedocs.io/en/latest/jupyterhub/customizing/user-storage.html?highlight=volume%2010GB#size-of-storage-provisioned

  2. The rest of the file system is "ephemeral" and linked to the K8s node configuration. At the time of setting this up, the default EBS volume attached to each node is 100GB https://github.com/terraform-aws-modules/terraform-aws-eks/blob/9022013844a61193a2f8764311fb679747807f5c/local.tf#L67 , so when you're on the hub you see:

    (notebook) jovyan@jupyter-scottyhq:~$ df -h
    Filesystem      Size  Used Avail Use% Mounted on
    overlay         100G  6.5G   94G   7% /
    tmpfs            64M     0   64M   0% /dev
    tmpfs            16G     0   16G   0% /sys/fs/cgroup
    /dev/nvme0n1p1  100G  6.5G   94G   7% /etc/hosts
    /dev/nvme3n1    9.8G  8.7G  1.1G  89% /home/jovyan

NOTE: with 4 users on the hub, if everyone is writing a bunch of data to a scratch location outside of /home/jovyan (like) /tmp, that root volume is shared, and also has some space taken up by k8s things. So for the case above we might run into trouble if everyone tries to write 30GB of data to /tmp (30*4=120GB > 94GB).