building 'web' in docker-compose took 8 minutes

[hshuaib90]

Is there a reason why it takes so long? Looking at the mlflow/Dockerfile it is pretty simple to build. May be worth putting in a warning in the README that i might take sometime first time

I did this locally on:

[tapjason]

The README mentions that "docker images that are not present on your local system will be pulled from dockerhub (which might take a while)", however, I'm not sure which part of the build report these images would be from. It seems that the parts taking the longest time to build are part of the docker python3.8 library, although I'm not sure how relevant that is.

[laurencejackson]

Thanks Jason, this is interesting. It might be that we could get away with a lightweight python image as opposed to the main image. When docker-compose builds 'web' it is building the Dockerfile at MLOPs/mlflow_server/mlflow/Dockerfile.

Could you try changing the python:3.8 image there to a lightweight python3.8 image? e.g. FROM python:3.8-slim

Once you've tried that could you rebuild and paste the same screenshot as you put here. Remember to delete your local cached Docker images or rebuild without cached images. Let me know if you encounter any errors or warnings with this approach.

[tapjason]

After changing to the lightweight python3.8 image, the package installation for python3.8 becomes much faster than before, however I get an extremely long error log for the psycopg2 package installation that would be too long to attach here, however it seems to center around a pg_config executable being unable to be located.

I added a line to install libpq-dev which contains the pg_config executable, and from there I reached the error shown below. Is there any pathing necessary to be added or am I making some fundamental mistake?

[laurencejackson]

Thanks Jason, this looks promising - could you try updating apt-get before you install libpq-dev?

RUN apt-get update && apt-get -y install libpq-dev

(The && chains together two commands in sequence and the -y flag prevents errors when apt-get asks for user confirmation)

[tapjason]

I added your line in place of what I had for installing libpq-dev, and I got the following error.

The error specifies a missing gcc compiler package and recommends installing psycopg2-binary, however I can't install with the typical line "apt-get -y install psycopg2-binary without running into a similar error from above "Unable to locate package psycopg2-binary".

I resorted to installing the gcc package directly with the dockerfile as follows:

This built 'web' successfully, taking 4~ minutes compared to the 10~ minutes it took locally before, however this should vary between different machines. The gcc package installation took 1 minute, so it might be worth looking into how to reduce unnecessary packages within this further as well as the 'nginx' build.

[laurencejackson]

Amazing, thanks Jason - I would say that a 6 minute reduction is good progress and sufficient to close this ticket. If we decide that the other docker-compose services (e.g. nginx) are taking too long we can raise other issues address them later.

To reduce the number of layers you can simply write: RUN apt-get update && apt-get -y install libpq-dev gcc to install libpq-dev and gcc in the same command.

Please could you raise a pull request to merge your branch into the develop branch and we can review and merge.