How to set up a Docker image using conda or mamba

[traversaro]

With @GiulioRomualdi and @CarlottaSartore, we had the doubt of the proper way of setting up a Docker image in a way that new commands "saw" the environment. Initially we followed https://pythonspeed.com/articles/activate-conda-dockerfile/, but the suggestion of using conda run was problematic as any COMMAND statement needed to be prefixed by conda run.

So, I think instead a good source of inspiration may be https://github.com/conda-forge/miniforge-images, that contains the Dockerfiles for the images published for example in https://hub.docker.com/r/condaforge/mambaforge . If feasible, it can make sense to use condaforge/mambaforge as a base image, while if that is not possible and you want to install conda/mamba manually it is a good idea to check how they do it in https://github.com/conda-forge/miniforge-images/blob/master/ubuntu/Dockerfile:

ARG MINIFORGE_NAME=Miniforge3
ARG MINIFORGE_VERSION=22.9.0-3
ARG TARGETPLATFORM

ENV CONDA_DIR=/opt/conda
ENV LANG=C.UTF-8 LC_ALL=C.UTF-8
ENV PATH=${CONDA_DIR}/bin:${PATH}

# 1. Install just enough for conda to work
# 2. Keep $HOME clean (no .wget-hsts file), since HSTS isn't useful in this context
# 3. Install miniforge from GitHub releases
# 4. Apply some cleanup tips from https://jcrist.github.io/conda-docker-tips.html
#    Particularly, we remove pyc and a files. The default install has no js, we can skip that
# 5. Activate base by default when running as any *non-root* user as well
#    Good security practice requires running most workloads as non-root
#    This makes sure any non-root users created also have base activated
#    for their interactive shells.
# 6. Activate base by default when running as root as well
#    The root user is already created, so won't pick up changes to /etc/skel
RUN apt-get update > /dev/null && \
    apt-get install --no-install-recommends --yes \
        wget bzip2 ca-certificates \
        git \
        tini \
        > /dev/null && \
    apt-get clean && \
    rm -rf /var/lib/apt/lists/* && \
    wget --no-hsts --quiet https://github.com/conda-forge/miniforge/releases/download/${MINIFORGE_VERSION}/${MINIFORGE_NAME}-${MINIFORGE_VERSION}-Linux-$(uname -m).sh -O /tmp/miniforge.sh && \
    /bin/bash /tmp/miniforge.sh -b -p ${CONDA_DIR} && \
    rm /tmp/miniforge.sh && \
    conda clean --tarballs --index-cache --packages --yes && \
    find ${CONDA_DIR} -follow -type f -name '*.a' -delete && \
    find ${CONDA_DIR} -follow -type f -name '*.pyc' -delete && \
    conda clean --force-pkgs-dirs --all --yes  && \
    echo ". ${CONDA_DIR}/etc/profile.d/conda.sh && conda activate base" >> /etc/skel/.bashrc && \
    echo ". ${CONDA_DIR}/etc/profile.d/conda.sh && conda activate base" >> ~/.bashrc

ENTRYPOINT ["tini", "--"]
CMD [ "/bin/bash" ]

It seems that the script is quite self-explanatory, in particular that in the activation script conda activate base should be prefixed by . ${CONDA_DIR}/etc/profile.d/conda.sh &&.

Once you have something that is working and you want to reduce its size, you can also check https://jcristharif.com/conda-docker-tips.html for additional tricks.

[diegoferigo]

You can take inspiration from the Docker image I'm using for my day-to-day development for few years now: diegoferigo/development-iit@Conda.

In particular, these lines create a default environment with micromamba.

The main trick to enable the environment in the rest of the Dockerfile (and I mean, in the build stage) is the combination of:

1. Enabling of the conda environment from the /etc/bash.bashrc file.
2. Usage of bash as shell during docker build as opposed to sh, performed in this line.

With this trick, every RUN command is executed in bash, that sources the file /etc/bash.bashrc that in turn it enables the conda environment.

[traversaro]

To include also alumnis, I guess also @prashanthr05 may have useful pointers.

[prashanthr05]

I have used what @traversaro has already mentioned in https://github.com/ami-iit/ami-commons/issues/5#issue-1531974087 as a reference to run conda and Docker and additionally took a lot of inspiration from @diegoferigo's repository https://github.com/diegoferigo/development-iit and @GiulioRomualdi's repository https://github.com/ami-iit/paper_romualdi_2022_icra_centroidal-mpc-walking to install all the necessary robotology-superbuild based dependencies using conda and self-activated conda environment at every docker run in https://github.com/ami-iit/paper_ramadoss_2022_humanoids_human-base-estimation.

But while cleaning up the conda installation for slimming down the images, it is better to keep an eye on what are we deleting to keep the image light, see issue https://github.com/prashanthr05/sample-docker-conda-cpp-project/issues/1#issuecomment-1283744226.

Some other useful resources to read through to keep images small, reduce build times, and an ounce of security while using Docker containers are mentioned in the following articles,

• https://towardsdatascience.com/slimming-down-your-docker-images-275f0ca9337e
• https://towardsdatascience.com/conda-pip-and-docker-ftw-d64fe638dc45
• https://snyk.io/blog/10-docker-image-security-best-practices/

[CarlottaSartore]

C.C. @mebbaid

[DanielePucci]

This seems to be important for many in @ami-iit/artificial-mechanical-intelligence dealing with Docker

[traversaro]

Possibly related to https://dev.to/crazy4pi314/how-to-get-the-best-conda-environment-experience-in-codespaces-4na9 .