search

jupyter-server / enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.
https://jupyter-enterprise-gateway.readthedocs.io/en/latest/
Other
615 stars 221 forks source link

Allow KIP to pull images relative to kernel provisioners #1197

Closed kevin-bates closed 1 year ago

kevin-bates commented 1 year ago

While working with the Remove Provisioners, I found that the Kernel Image Puller (KIP) was not pre-pulling images configured for provisioners. Since their kernel spec configurations are nearly identical wrt how images are defined (process proxies use a process_proxy stanza while provisioners use a kernel_provisioner stanza), it's trivial for KIP to support both.

This PR adds code to attempt to access the kernel_provisioner stanza when the process_proxy stanza is not found in a given kernel spec.

kevin-bates commented 1 year ago

Since this is not necessarily straightforward to test (I use a Kernel Gateway image running in K8s since it can deal with kernel provisioners) here is some output from KIP:

[I 2022-11-15 00:21:54,058 kernel_image_puller.MainThread] Fetching kernelspecs from 'http://kernel-gateway.kernel-gateway:8888/api/kernelspecs' ...
Mon, Nov 14 2022 4:21:54 pm | [I 2022-11-15 00:21:54,316 kernel_image_puller.t1] Task received to pull image: elyra/rp-kernel-scala:dev
Mon, Nov 14 2022 4:21:54 pm | [I 2022-11-15 00:21:54,316 kernel_image_puller.t2] Task received to pull image: elyra/rp-kernel-r:dev
Mon, Nov 14 2022 4:21:54 pm | [D 2022-11-15 00:21:54,341 kernel_image_puller.t1] Checked existence of image 'elyra/rp-kernel-scala:dev' in 0.024 secs. exists = False
Mon, Nov 14 2022 4:21:54 pm | [I 2022-11-15 00:21:54,341 kernel_image_puller.t1] Pulling image 'elyra/rp-kernel-scala:dev'...
Mon, Nov 14 2022 4:21:54 pm | [D 2022-11-15 00:21:54,346 kernel_image_puller.t2] Checked existence of image 'elyra/rp-kernel-r:dev' in 0.027 secs. exists = False

Prior to this change, all you'd see is short, periodic queries against the service. (The queries use a shorter period (rather than 5 minutes) because no images have been pulled. We only increase the delay to the configured value (300 seconds) once images have been found.)

[I 2022-11-15 00:21:54,058 kernel_image_puller.MainThread] Fetching kernelspecs from 'http://kernel-gateway.kernel-gateway:8888/api/kernelspecs' ...

Here's the set of chart files I use to deploy kernel-gateway. I've also pushed the a dev image to elyra/kernel-gateway:dev: kernel-gateway-3.1.0-dev0.tgz (The version information can be ignored.)

You should be able to deploy using the following...

kubectl create namespace kernel-gateway
helm  upgrade --install  kernel-gateway https://github.com/jupyter-server/enterprise_gateway/files/10013790/kernel-gateway-3.1.0-dev0.tgz -n kernel-gateway