search

Arxcis / adventofcode2020

Community-project solving https://adventofcode.com/ - problems, with Github CI, Docker and support for many languages.
6 stars 8 forks source link

Docker image size is getting too big #126

Closed Arxcis closed 2 years ago

Arxcis commented 2 years ago

Stats

After #125 - adding Kotlin support - the biggest layer is now 3.8GB!

[jonas@macbookpro adventofcode2020]$ make push
docker push "jonasjso/adventofcode2020:2021-12-04-with-kotlin-1.5.32"
The push refers to repository [docker.io/jonasjso/adventofcode2020]
23ad0c1ac26c: Pushing [==>                                                ]  213.3MB/3.8GB
But on Dockerhub it looks a bit smaller, still it is increasing in size 2020-12-18-with-zig-v0.7.1 2021-12-04-with-kotlin-1.5.32
image image

Ideas on how to reduce size

Some ideas off the top of my head:

Avokadoen commented 2 years ago

Idea: Although not a direct solution, but maybe it would alleviate some of the pain to use some sort of sibling hierarchy. If you want to run all tests you communicate with the instance that will delegates to the zig, kotlin, go, etc .. -container. This way when a new update to one of the containers is applied then it will not produce an artifact containing GBs of data.

Of course this comes at the cost of execution time and project complexity ... 🤔

Avokadoen commented 2 years ago

github workflow even support this to some extent

Avokadoen commented 2 years ago

This could also solve #129 with a zig0.7.1 and a zig0.9-nigthly image

Arxcis commented 2 years ago

@Avokadoen Good idea 👏 Though I am a bit surprised/scared of the potential that the complexity of this project may spiral out of control 😆 I was not prepared for this when starting back in 2020 😅

Arxcis commented 2 years ago

Building on the discussion in this thread, and the one in #129, I am seeing two paths we could go down:

~With Path 2, we also need images per year, because the issue discussed in #129 will not go away. So we will potentially end up with a lot more docker images with path 2, as with path1 we are only producing 1 new image per year. The potential for Path 2 is, in worst case, to produce N new images per year, where N is the number of languages.~ False claim, see comment below

Avokadoen commented 2 years ago

I don't think path 2 would not require year based images if image is based on version as well as language like mentioned in my previous comment 🤔

Arxcis commented 2 years ago

I don't think path 2 would not require year based images if image is based on version as well as language like mentioned in my previous comment 🤔

Fair point. Dockerhub will keep the history of images for a given language. And different solutions from different years may point to different versions of the same image. Aha! Ok. Then It is not as bad as I feared 😄

Avokadoen commented 2 years ago

Path 1 seems simpler like you mentioned and there would be no need to keep languages that are not used in the next year and vice versa which would help reduce image size

Arxcis commented 2 years ago

Fair point. Dockerhub will keep the history of images for a given language. And different solutions from different years may point to different versions of the same image. Aha! Ok. Then It is not as bad as I feared 😄

By the way, doesn't this actually also solve #129 , because the 2020.yaml-workflow points to a specific tagged-version of our docker-image. We could just NOT update this tag in the 2020 workflow and thus updating zig to a newer version will in fact not break 2020 solutions, because the 2020-workflow use an older tag of our docker image... 🤔 While the 2021-workflow may use the newer tag! 🥳

Avokadoen commented 2 years ago

Stumbling on solutions is the best way to solve problems hahah

Arxcis commented 2 years ago

Docker image size much smaller after PR #138 : image