search

RhodiumGroup / docker_images

Docker images for Rhodium's jupyterlab deployments
2 stars 2 forks source link

pyTC and master are diverging a whole lot #136

Open delgadom opened 4 years ago

delgadom commented 4 years ago

How do we ensure the jupyter environment stays consistent across dependent images? @bolliger32

bolliger32 commented 4 years ago

hmm...not sure off the top of my head. Do we need them to stay consistent? I thought one of the motivations for allowing separate images is that we don't necessarily need to have a single master image, so I haven't really been trying to keep dev-pyTC base consistent with master.

If we wanted to keep them consistent, one way would be to have a separate repo that builds a base image with everything that we want to stay consistent, and then another repo that has a stack of images (including the main one, as well as any customized ones) which each build from the base image.

delgadom commented 4 years ago

yeah they don't necessarily, but changing around the directory structure in pyTC really leans into their divergence. jupyter versions definitely need to be in sync, as we need to always maintain sync between all images and the jupyterhub cluster version. and it would be ideal to keep as much as possible of the user directory setup in sync so we don't have artifacts moving around from users switching between images. I was imagining an eventual split into a base image that sets up jupyter with a bare minimum env, but with all the user directory setup. then we could build rhodium/notebook on top of that.

bolliger32 commented 4 years ago

Yeah that sounds like a reasonable way to do it. If we want to maintain some level of sync-ness, I think having a dedicated base image will help define exactly what we want to keep in sync vs. leaving it up for interpretation. I think as of now jupyterhub and the user directory manipulation should be in sync btwn dev-pyTC-base and master but let me know if you spotted something that looked like that was not true and I'll give it a fix.

brews commented 4 years ago

Is the main difference just in python packages or is there a larger difference in setup or compiled dependencies that creates this divergence?

I like the idea of a vanilla Rhodium image and then allowing extensions on this for special cases.

bolliger32 commented 4 years ago

there's definitely some differences all around. There were some non-python capabilities that needed to be implemented and once I started implementing those I wasn't necessarily trying to make sure everything stayed consistent with the master branch.