Track and test upgrade of pangeo-notebook image in veda-jh-environments

abarciauskas-bgse commented 10 months ago

Description

We would like to upgrade the pangeo-notebook image used by the VEDA JupyterHub so we get an updated version of xarray to solve an issue with reading chunks from h5netcdf (see https://github.com/pydata/xarray/issues/8691). This may also resolve some of the features of interest in https://github.com/NASA-IMPACT/veda-analytics/issues/94.

@wildintellect mentioned there was an issue with moving to JupyterLab 4, which is a part of this updated pangeo-notebook image https://github.com/pangeo-data/pangeo-docker-images/blob/2024.01.23/pangeo-notebook/packages.txt#L270C1-L270C19

Checklist:

[x] Open PR (in this repo) to upgrade the pangeo-notebook image tag: https://github.com/NASA-IMPACT/veda-jh-environments/blob/main/docker-images/base/pangeo-notebook/Dockerfile#L1 to have the updated image tag pangeo/pangeo-notebook:2024.01.23
[x] Open a PR (in this repo) to upgrade the custom singleuser image that points to the new tag of the base image (output of step 1 above). NOTE: We need to do this b/c ONBUILD commands in docker are only heritable for the immediate child. And our "custom" singleuser images will include dependencies that we want to be picked up with the adjacent environment.yml
[x] Once you have custom image tag (from step 2 above), test it by simply pasting it into the 'bring your own image' field of the profile. Test the xarray feature mentioned as well as running through a few other notebooks to make sure the hub is working as expected.
[x] Once the PR to veda-jh-environments is merged, and the image is built in ECR, open a PR which updates the hash for the nasa-veda-singleuser image here: https://github.com/2i2c-org/infrastructure/blob/2a17dfb1d0fc2cfc5d6202cd2af582e090fdabce/config/clusters/nasa-veda/common.values.yaml#L70
[x] Request 2i2c deploy the VEDA jupyterhub with the updated nasa-veda-singleuser image tag with evidence of the working image

ranchodeluxe commented 9 months ago

@abarciauskas-bgse :

added a second checklist step that was missing
I changed the order of steps 3/4
the last two steps are codified in this documentation already: https://github.com/NASA-IMPACT/veda-jh-environments/blob/main/docs/2i2c_singleuser_image.md

abarciauskas-bgse commented 9 months ago

@ranchodeluxe thanks for the updates. Perhaps we should merge these instructions into that documentation and / or create an issue template for future upgrades.

wildintellect commented 9 months ago

@abarciauskas-bgse are you going to do the PR to bump the pangeo image? the stac_ipyleaflet fix is a simple version bump too (current version is broken), and pypi install of lonboard should be straight forward if @kylebarron finished the new release. But I don't know if we want to mix all these things. Or test one at a time, then do the PR to 2i2c with everything?

kylebarron commented 9 months ago

I'm hoping to release a new minor version of lonboard tomorrow (depending on time around zarr sprint)

abarciauskas-bgse commented 9 months ago

@wildintellect

are you going to do the PR to bump the pangeo image?

yes, thanks for the nudge: https://github.com/NASA-IMPACT/veda-jh-environments/pull/42

wildintellect commented 8 months ago

@ranchodeluxe do it

ranchodeluxe commented 8 months ago

PR in: https://github.com/2i2c-org/infrastructure/pull/3788

NASA-IMPACT / veda-jh-environments

Track and test upgrade of pangeo-notebook image in veda-jh-environments #41

Description

Checklist: