Long build time for docker

[laurencejackson]

The first time you build the docker container which is used to run the project through mlflow the build takes a very long time. This is probably to do with including a lot of extra libraries through cuda/ubuntu images. Shoudl investigate whether we can slim this down.

Relevant file is the Dockerfile at root.

[laurencejackson]

Best option will probably be to build a docker container with cuda, torch, git etc ready and push it to dockerhub. Any developer can then pull that image with everything ready - makes no sense to install so many things each time.

[laurencejackson]

[hshuaib90]

Best option will probably be to build a docker container with cuda, torch, git etc ready and push it to dockerhub. Any developer can then pull that image with everything ready - makes no sense to install so many things each time.

Yes I experience this same issue. Surely such a docker image already exists? Do NVIDIA have one? Maybe on NGC?

[laurencejackson]

Yeah there's one on NGC, I found it this morning! It's enormous but will be cached anyway.

[laurencejackson]

The right tag is :20.08-py3 so the Dockerfile will need to be updated to look like:

FROM nvcr.io/nvidia/pytorch:20.08-py3

# Configure application
ADD requirements.txt .
RUN python3 -m pip install -r requirements.txt

# Setup MLOps
RUN git clone --branch feature/mlflow_server https://github.com/GSTT-CSC/MLOps.git
RUN python3 -m pip install -r MLOps/requirements.txt \
    && python3 -m pip install MLOps/

[laurencejackson]

Closing since we have moved onto a different Dcokerfile now with a pytorch base image. Long build time may also have been associated with large build contexts so ensure that the project directory (or Dockerfile directory) contain only the project code. Alternatively add files to .dockerignore