os-climate / os_c_data_commons

Repository for Data Commons platform architecture overview, as well as developer and user documentation
Apache License 2.0
21 stars 10 forks source link

Jupyterhub cannot get sizing details for provisioning notebooks #291

Open Gregory-Pereira opened 1 year ago

Gregory-Pereira commented 1 year ago

Stalls notebook provisioning to an unreasonable degree. Google networking console reveales 403 cannot get resource sizing reqs for a new notebook. Affects all notebook types but demoed with elyra.

Follow on issue: opening a bunch of jupyterhub notebooks simultaneous throws 502. Will look into this after inital steps are taken to search for this issue in the upstream (assign @MichaelTiemann).

/assign

Gregory-Pereira commented 1 year ago

So interestingly, the first network request actually gets all the sizing data in a comfigmap and then makes a request to each individual sizing configuration, however despite returning with a 403 you the URL it used does indeed reach the configurations listed.

Gregory-Pereira commented 1 year ago

Seems that the deployment is bassed off the replica controller which stores the full encoded deployment config in metadata in the openshift.io/encoded-deployment-config annotation. This looks for the Jupyterhub-cfg configmap. From the deployment config:

- name: SINGLEUSER_PVC_SIZE
              valueFrom:
                configMapKeyRef:
                  name: jupyterhub-cfg
                  key: singleuser_pvc_size
Gregory-Pereira commented 1 year ago

Cross reference our deployment with official docs: https://github.com/opendatahub-io-contrib/jupyterhub-singleuser-profiles

https://console-openshift-console.apps.odh-cl2.apps.os-climate.org/search/ns/odh-jupyterhub?kind=core%7Ev1%7EConfigMap&q=jupyterhub%3Dsingleuser-profiles

Gregory-Pereira commented 1 year ago

This is the request it makes to get the value: https://jupyterhub-odh-jupyterhub.apps.odh-cl2.apps.os-climate.org/services/jsp-api/api/user/configmap