I've re-built our gardenai/base images to derive from the official python images and no longer include conda; the only important changes here are just pointing the constants to the new base image tags. I also included the script I used to build/push the images.
Previously, base images started from python:{version}-slim, then installed a few utilities like wget and git, and then finally installed/configured miniconda.
The new images opt for python:{version} images instead of the respective *-slim variants, because (per the official dockerhub overview) when using a *-slim image:
pip install may fail when installing a Python distribution package from a source distribution. This image does not contain the Debian packages required to compile extension modules written in other languages
which I suspect was a major contributing factor to #440.
The only trade-off is that the new base images are basically the same size as the old ones, without conda. This doesn't really matter imo because the lion's share of the final image size is going to be user packages anyways.
Discussion
I made some tweaks to existing code, but I decided to keep the logic we already have that would e.g. install a conda environment.yml if the user passed one as a --requirements file. This would now fail if using one of our new base images, but power-users who really need conda can still pass a --custom-image-uri instead of one of our convenient base images and it'll work like it would have using the old images.
In the meantime, I don't want to support conda base images if we don't have to, and at the moment it seems like we don't have to.
Testing
Same manual testing I did for #436 -- I have a dummy entrypoint that just reports the currently-installed version of dill. published with the new 3.11 images, that entrypoint returns 0.3.6 like we hoped it would instead of crashing the entire pod like described in #440.
Documentation
no docs, but I did remove mentions of conda from the help text for the various notebook commands, so as not to encourage bad behavior.
closes #439 and (seemingly 🤞) fixes #440.
Overview
I've re-built our gardenai/base images to derive from the official python images and no longer include conda; the only important changes here are just pointing the constants to the new base image tags. I also included the script I used to build/push the images.
Previously, base images started from
python:{version}-slim, then installed a few utilities like wget and git, and then finally installed/configured miniconda.The new images opt for
python:{version}images instead of the respective*-slimvariants, because (per the official dockerhub overview) when using a*-slimimage:which I suspect was a major contributing factor to #440.
The only trade-off is that the new base images are basically the same size as the old ones, without conda. This doesn't really matter imo because the lion's share of the final image size is going to be user packages anyways.
Discussion
I made some tweaks to existing code, but I decided to keep the logic we already have that would e.g. install a conda environment.yml if the user passed one as a
--requirementsfile. This would now fail if using one of our new base images, but power-users who really need conda can still pass a--custom-image-uriinstead of one of our convenient base images and it'll work like it would have using the old images.In the meantime, I don't want to support conda base images if we don't have to, and at the moment it seems like we don't have to.
Testing
Same manual testing I did for #436 -- I have a dummy entrypoint that just reports the currently-installed version of dill. published with the new 3.11 images, that entrypoint returns 0.3.6 like we hoped it would instead of crashing the entire pod like described in #440.
Documentation
no docs, but I did remove mentions of conda from the help text for the various notebook commands, so as not to encourage bad behavior.
📚 Documentation preview 📚: https://garden-ai--446.org.readthedocs.build/en/446/