Jupyterhub base image for the NASA Openscapes Hub
In collaborative efforts -like this NASA hackathon- there are multiple teams working on different stacks and we often run into situations where Team A
will need to use Python 3.8 with say xarray v0.14
and Team B
may need Python 3.9 and xarray v0.17
. A simple solution would be to reconcile these 2 environments so both teams can run their code. However, this is not always straight forward or even possible. Therefore having a multi kernel base image for Jupyterhub deployments makes a lot of sense.
corn
uses the amazing Pangeo's base image, installs all the environments it finds under ci/environments
and makes them available as kernels in the base image so users can select which kernel to use depending on their needs. The only requirement to add kernels is to use a conda environment.yml file (pip dependencies can be included in environment.yml) and a name file.
To add a new kernel we need to create a new folder under ci/environments/
and add the 2 files described above. Say we want to run our amazing new notebook that uses pandas and python 3.10.
We will need a conda environment file environment.yml
name: amazing-env
channels:
- conda-forge
dependencies:
- python="3.10"
- pandas>=1.3
- pip
and our name.txt file
amazing-env
### Updating quarto To update the [quarto](https://quarto.org) installation you'll need to change the version number in corn's [Dockerfile](https://github.com/NASA-Openscapes/corn/blob/main/ci/Dockerfile#L9). After committing changes, the GitHub Action will begin - see next. ## Updating the image in the JupyterHub After we commit our changes to the `main` branch of this repo, the GitHub Action build will be triggered. Then, the Github Action will push the resulting Docker image to [dockerhub](https://hub.docker.com/r/openscapes/python/tags), creating an image tagged with the commit hash. This can take ~20 minutes. You can try this newly created image by using the "Bring your own image" functionality in the JupyterHub. Specify the image with the name `openscapes/python:$TAG`, where `$TAG` is the tag of the Dockerhub image (which is the same as the commit hash). You can copy the name from the `docker pull` command shown in [Dockerhub](https://hub.docker.com/r/openscapes/python/tags). Once you've verified it is working the way you want, we need to update the python image in our Jupyterhub configuration. The quickest way to do this is to create a pull request [here](https://github.com/2i2c-org/infrastructure/blob/main/config/clusters/openscapes/common.values.yaml#L68), updating `openscapes/python:$TAG`, with the tag/commit hash. For 2i2c deployments there is a GUI that allows administrators to do it. Then, you'll go toNote: if you have pip installable dependencies, they must be listed using a
requirements.txt
file.