search

pangeo-data / pangeo-cloud-federation

Deployment automation for Pangeo JupyterHubs on AWS, Google, and Azure
https://pangeo.io/cloud.html
58 stars 32 forks source link

Discussion: Image Management & Staging -> Prod workflow #692

Open scottyhq opened 4 years ago

scottyhq commented 4 years ago

This is an issue for discussing the current staging -> prod workflow and management and testing of Docker images.

The original design of hubploy works great for a single image per hub, defined in a subfolder. This is currently the configuration for the GCP hub: https://github.com/pangeo-data/pangeo-cloud-federation/tree/staging/deployments/gcp-uscentral1b/image/binder

Over time on the aws-uswest2.pangeo.io we've found it helpful to have a handful of different images available for selection. In particular, pangeo/ml-notebook which has specific GPU config https://github.com/pangeo-data/pangeo-cloud-federation/blob/7e842bf024d0aa90d42c8f0e3a6fa79bc0e4b91f/deployments/icesat2/config/common.yaml#L31-L39

Or images for special events like hackweeks: https://github.com/pangeo-data/pangeo-cloud-federation/blob/7e842bf024d0aa90d42c8f0e3a6fa79bc0e4b91f/deployments/icesat2/config/common.yaml#L19-L22

A while back I started using the latest tag in the profile list because I couldn't be bothered updating tags frequently. This works pretty well if the 'latest' tag is always guaranteed as a tested production release. Which is the goal with our pangeo-docker-images repository design (https://github.com/pangeo-data/pangeo-docker-images#image-tagging-and-continuous-building). This key to using latest tags in config is to ensure that the pod always re-pulls the image (default is 'IfNotPresent') when launched:

    singleuser:
      image:
         pullPolicy: 'Always'

This still isn't ideal from a reproducibility standpoint (probably best to use explicit tags).

Possible next steps:

1) Tackle this open issue on hubploy for multiple images support https://github.com/yuvipanda/hubploy/issues/10 ?

2) Add GitHub Action to watch for new tags on DockerHub and issue PRs to bump all image tags under profile list?

salvis2 commented 4 years ago

This would be really nice. We could probably quash the issue we had a bit ago with https://github.com/yuvipanda/hubploy/issues/73 and have some listed images from DockerHub, some from ECR, etc.