search

NASA-IMPACT / pangeo-notebook-veda-image

Docker container based on pangeo-notebook used on VEDA JupyterHub
https://gallery.ecr.aws/nasa-veda/pangeo-notebook-veda-image
MIT License
0 stars 2 forks source link

update pangeo base image to 2024.06.28 #11

Open batpad opened 1 month ago

batpad commented 1 month ago

refs https://github.com/NASA-IMPACT/veda-jupyterhub/issues/40

Upgrade to latest pangeo base image to test with.

cc @jsignell @sunu

batpad commented 1 month ago

There seem to be minor diffs in the outputs that are failing the tests: https://github.com/NASA-IMPACT/pangeo-notebook-veda-image/actions/runs/9759072270/job/26935002169

This makes me very happy haha :-)

@jsignell - would you like to take a look? Am also happy to regenerate the outputs snapshot and go ahead with this -- looking at the result diff, this definitely seems totally okay.

And thank you again @jsignell - having these tests in place makes me feel SO much better about these updates ..

jsignell commented 1 month ago

This PR would just push the image to ecr right? It wouldn't make this image the default one on veda jupyterhub or anything right?

If that is the case then yeah I feel fine with you regenerating the output so that we can build the image and then we can do a more thorough test once it's available just since there are some big version upgrades.

batpad commented 1 month ago

This PR would just push the image to ecr right? It wouldn't make this image the default one on veda jupyterhub or anything right?

That's correct - pushes to this repo just build and push the images to ECR - it still requires a manual step to replace the image tags in the infrastructure repo for them to get deployed as the default images on the hub :-)

I tried regenerating the outputs - hopefully the tests will pass now and you'll have an image to test with :)

batpad commented 1 month ago

@sunu - I tried regenerating the outputs here and re-running the tests, but am still seeing a failure.

Do you have any idea of what's going on here? :)

batpad commented 1 month ago

@sunu thank you! Does it make sense to add this --nb-force-regen to the docs or so?

batpad commented 1 month ago

@jsignell image is up on ECR - so you can use the Bring Your Own Image option with public.ecr.aws/nasa-veda/pangeo-notebook-veda-image:92fe086baf92 to test this image on the hub.

jsignell commented 1 month ago

Just want to point out that this version of the pangeo-notebook docker image (as well as the 2024-07-02 version) is still on numpy 1.26. So the geopandas bump to 1.0 is a big deal, but there isn't as much risk as we maybe expected since numpy is still under 2.

batpad commented 1 month ago

but there isn't as much risk as we maybe expected since numpy is still under 2.

Aha! I should have looked closer - I thought this was bumping numpy to 2 ..

@jsignell will follow your advice for whether we should do this update, or wait for a new pangeo base, or however you feel is best.

jsignell commented 1 month ago

I'm running through the docs notebooks to just double check, but my gut is we can probably just do this update without making it any easier for people to use old versions of the image.

jsignell commented 1 month ago

I am having a really weird time with jupyterlab in that image. Hard to say if it's a totally new problem, but I am seeing really jittery scroll and I can easily get to places in the notebook that haven't aren't rendered at all. I also found a few hvplot and maybe rasterio bugs that sidetracked me. So I didn't quite finish evaluating.

batpad commented 1 month ago

@jsignell I see there's already an updated version 2024.07.02 - should we try just updating to that? It seems like it is still on numpy<2