search

errbotio / errbot

Errbot is a chatbot, a daemon that connects to your favorite chat service and bring your tools and some fun into the conversation.
http://errbot.io
GNU General Public License v3.0
3.08k stars 607 forks source link

Optimize Dockerfile #1679

Closed amayer5125 closed 3 days ago

amayer5125 commented 5 months ago

Use a virtual environment to make copying packages from build stage to the final stage easier. It also reduces the image size by 94Mb.

I used python:3-slim as the final image. Previously we were using python:3.9-slim. I think using the newer python version is a good idea, but I can change it back if we don't want to use the newer version.

Size comparison:

REPOSITORY        TAG       IMAGE ID       CREATED         SIZE
errbotio/errbot   master    416f2383b24a   7 seconds ago   325MB
errbotio/errbot   new       0226c1a39cd1   2 minutes ago   231MB
sijis commented 5 months ago

Thank you for you suggestion on improvement.

I try and not to modify the default PATH variable, as it changes the expected location of files. Do you think there could be another approach in reducing the files? Could the wheel files be related (even though directory is removed)?

amayer5125 commented 5 months ago

The PATH changes I make are the same thing a python virtual environment does when you run the activate script. I do append the existing PATH after the virtual environment location so any commands outside of the virtual environment should still work as expected.

The size difference almost certainly has to do with the wheel files from the COPY --from=build /wheel /wheel line. That line copies the files to the image and creates a new layer. The subsequent line starts a new layer and runs rm -rf /wheel which only deletes the files in the new layer. They still exist in the previous layer though, so they contribute to the overall size of the image. I cannot think of a way to copy the files into the new image for the duration of the install and then delete them.

sijis commented 5 months ago

I see what you mean.

Quickly looking, there may be something with RUN --mount that is suggested https://stackoverflow.com/a/76670359, which sounds promising. I have yet to try it myself though.

amayer5125 commented 5 months ago

I did some reading on --mount. If you do not want to use virtual environments for the image I think it is a great alternative to reduce the size. Switching from COPY to --mount only saved about 21MB. I did some of the other refactors I did in this pull request and got the image size down way farther.

Here are my test results:

REPOSITORY         TAG              IMAGE ID       CREATED          SIZE
errbotio/errbot    venv             e2ceb54767df   34 seconds ago   216MB
errbotio/errbot    mount-refactor   1f06e5451f5d   12 minutes ago   211MB
errbotio/errbot    mount            cb5f34d5318e   25 minutes ago   305MB
errbotio/errbot    master           6d3c52d7d1b6   34 minutes ago   326MB

master

Built off of master with no modifications.

mount

Remove COPY --from=build /wheel /wheel line and add --mount=from=build,source=/wheel,target=/wheel to the pip install line.

mount-refactor

Same as mount tag, plus removed apt install git, use python:3.9 image for build, and remove unneeded FROM base.

venv

This current pull request built with python:3.9 build stage and python:3.9-slim final image.

sijis commented 3 days ago

@amayer5125 Thank you for all the work with this. merging.