jupyter-server / enterprise_gateway

A lightweight, multi-tenant, scalable and secure gateway that enables Jupyter Notebooks to share resources across distributed clusters such as Apache Spark, Kubernetes and others.
https://jupyter-enterprise-gateway.readthedocs.io/en/latest/
Other
617 stars 222 forks source link

Use local kernel images if they exist #1143

Closed kevin-bates closed 1 year ago

kevin-bates commented 1 year ago

Prior to these changes, the Kernel Image Puller would unconditionally pull images on its first pass when the policy was IfNotPresent despite the fact that applicable images are locally present. This prevented the ability to use locally-defined images - which is useful in dev environments, but also in environments in which you wish "pre-seed" images.

There are two areas of change.

  1. The KIP has been updated to check for image existence even when the image has yet to be pulled (i.e., the first-time scenario). If the image exists, it considers it pulled and adds it to KIP's pulled-images set. KIP will continue to monitor the images, just as it does today.
  2. The helm chart has been modified to tolerate a missing Values.kip.defaultContainerRegistry value in which case the corresponding env KIP_DEFAULT_CONTAINER_REGISTRY will not be set when KIP is started.
    a. Related to this, KIP now defaults the non-existence of KIP_DEFAULT_CONTAINER_REGISTRY to the empty string "", which then prevents a registry-identifier from being prefixed to the image name if the kernel.json doesn't already include one.

Change 2 just makes it easier to convey that no registry should be used. However, it should be noted (probably when using the Docker runtime) that docker.io is still the default registry when the empty string is in play. I believe this is a function of the DockerClient used to pull images.

In my testing, even when the default registry was docker.io local images named elyra/kernel-py:dev would still be used. If those images did not exist or were removed while KIP was running, KIP will pull docker.io/elyra/kernel-py:dev. However, when quay.io was configured as KIP_DEFAULT_CONTAINER_REGISTRY, local images will be pulled from quay.io unless the local image included the quay.io prefix (e.g., quay.io/elyra/kernel-py:dev).

kevin-bates commented 1 year ago

Thanks for the review @kiersten-stokes!