Closed tsluyter closed 3 years ago
As per the comments in my pull request about shrinking: I've rethought our Docker strategy.
By defining "skf-labs/alpine37" we can make a common underlayer that applies to almost every container we make. This means that students will only need to "pull" that layer once, in order to build 1-N containers.
This approach means that the final container space grows a little, but in reality the storage space used will be even less than before! The big underlayers will be common/shared by all containers.
Here's a comparison between two Docker containers built with the current Dockerfiles, and two with my adjusted Dockerfiles.
Both builds have roughly 100 MB in common: the base OS plus Python3... The big difference lies in the git clone and cleanup afterwards. Lines starting with ** are what the containers have in common.
=== Original config ===
$ docker images | grep owasp-skf-lab
blabla1337/owasp-skf-lab cors d38a24f88636 4 months ago 331MB
blabla1337/owasp-skf-lab xss 00444b7ad9e2 11 days ago 338MB
$ docker history blabla1337/owasp-skf-lab:cors
IMAGE CREATED CREATED BY SIZE COMMENT
d38a24f88636 4 months ago CMD ["python3" "./CORS.py"] 0B buildkit.dockerfile.v0
<missing> 4 months ago RUN /bin/sh -c pip3 install -r requirements.… 8.09MB buildkit.dockerfile.v0
<missing> 4 months ago WORKDIR /skf-labs/CORS 0B buildkit.dockerfile.v0
<missing> 4 months ago RUN /bin/sh -c git clone https://github.com/… 221MB buildkit.dockerfile.v0
**<missing> 4 months ago RUN /bin/sh -c apk update --no-cache && apk … 97.9MB buildkit.dockerfile.v0
**<missing> 4 months ago MAINTAINER Glenn ten Cate <glenn.ten.cate@ow… 0B buildkit.dockerfile.v0
**<missing> 19 months ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
**<missing> 19 months ago /bin/sh -c #(nop) ADD file:aa17928040e31624c… 4.21MB
$ docker history blabla1337/owasp-skf-lab:xss
IMAGE CREATED CREATED BY SIZE COMMENT
00444b7ad9e2 11 days ago CMD ["python3" "./XSS.py"] 0B buildkit.dockerfile.v0
<missing> 11 days ago RUN /bin/sh -c chmod +x XSS.py # buildkit 0B buildkit.dockerfile.v0
<missing> 11 days ago RUN /bin/sh -c pip3 install -r requirements.… 8.17MB buildkit.dockerfile.v0
<missing> 11 days ago WORKDIR /skf-labs/XSS 0B buildkit.dockerfile.v0
<missing> 11 days ago RUN /bin/sh -c git clone https://github.com/… 228MB buildkit.dockerfile.v0
**<missing> 11 days ago RUN /bin/sh -c apk update --no-cache && apk … 97.9MB buildkit.dockerfile.v0
**<missing> 11 days ago MAINTAINER Glenn ten Cate <glenn.ten.cate@ow… 0B buildkit.dockerfile.v0
**<missing> 19 months ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
**<missing> 19 months ago /bin/sh -c #(nop) ADD file:aa17928040e31624c… 4.21MB
=== My proposed config ===
$ docker images | grep "skf-labs/"
skf-labs/cors latest 7e0b8fd2c881 23 minutes ago 112MB
skf-labs/xss latest d39fd0589ab2 32 minutes ago 112MB
$ docker history skf-labs/cors
IMAGE CREATED CREATED BY SIZE COMMENT
7e0b8fd2c881 27 minutes ago CMD ["python3" "./CORS.py"] 0B buildkit.dockerfile.v0
<missing> 27 minutes ago RUN /bin/sh -c find . -name "*.sh" -o -name … 850kB buildkit.dockerfile.v0
<missing> 27 minutes ago WORKDIR /skf-labs/CORS 0B buildkit.dockerfile.v0
<missing> 27 minutes ago RUN /bin/sh -c cd /tmp && mkdir /skf-labs &&… 1.69MB buildkit.dockerfile.v0
<missing> 27 minutes ago RUN /bin/sh -c wget https://raw.githubuserco… 7.1MB buildkit.dockerfile.v0
**<missing> 40 minutes ago MAINTAINER Glenn ten Cate <glenn.ten.cate@ow… 0B buildkit.dockerfile.v0
**<missing> 40 minutes ago RUN /bin/sh -c apk add dos2unix --no-cache -… 484kB buildkit.dockerfile.v0
**<missing> 45 minutes ago RUN /bin/sh -c apk update --no-cache && apk … 97.9MB buildkit.dockerfile.v0
**<missing> 45 minutes ago MAINTAINER Glenn ten Cate <glenn.ten.cate@ow… 0B buildkit.dockerfile.v0
**<missing> 19 months ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
**<missing> 19 months ago /bin/sh -c #(nop) ADD file:aa17928040e31624c… 4.21MB
$ docker history skf-labs/xss
IMAGE CREATED CREATED BY SIZE COMMENT
d39fd0589ab2 35 minutes ago CMD ["python3" "./XSS.py"] 0B buildkit.dockerfile.v0
<missing> 35 minutes ago USER www 0B buildkit.dockerfile.v0
<missing> 35 minutes ago RUN /bin/sh -c find . -name "*.sh" -o -name … 1.26MB buildkit.dockerfile.v0
<missing> 40 minutes ago WORKDIR /skf-labs/XSS 0B buildkit.dockerfile.v0
<missing> 40 minutes ago RUN /bin/sh -c cd /tmp && mkdir /skf-labs &&… 1.62MB buildkit.dockerfile.v0
<missing> 40 minutes ago RUN /bin/sh -c wget https://raw.githubuserco… 6.82MB buildkit.dockerfile.v0
**<missing> 40 minutes ago MAINTAINER Glenn ten Cate <glenn.ten.cate@ow… 0B buildkit.dockerfile.v0
**<missing> 40 minutes ago RUN /bin/sh -c apk add dos2unix --no-cache -… 484kB buildkit.dockerfile.v0
**<missing> 45 minutes ago RUN /bin/sh -c apk update --no-cache && apk … 97.9MB buildkit.dockerfile.v0
**<missing> 45 minutes ago MAINTAINER Glenn ten Cate <glenn.ten.cate@ow… 0B buildkit.dockerfile.v0
**<missing> 19 months ago /bin/sh -c #(nop) CMD ["/bin/sh"] 0B
**<missing> 19 months ago /bin/sh -c #(nop) ADD file:aa17928040e31624c… 4.21MB
You have addressed this issue for most containers through your own branches and methods.
You may have noticed pull request 115 -> https://github.com/blabla1337/skf-labs/pull/115
I've been going over a few of your Docker files to make sure that the containers we produce are as small as possible. For students with limited resources (young students, developing countries, little expendable income) storage space can come at a premium. Shrinking our requirements by 60% seems like a very user-friendly thing to do.
With my proposed refactoring, our resulting containers go down from an average 330MB to ~77MB.
The biggest factor in the current ~330MB size per container, lies in the fact that the whole SKF Labs Git repo is pulled into the container and then never removed. By refactoring the Dockerfile to git pull, and then only copying the needed content, it's a huge space-saver already.
Another nice way to save space on our layers is to combine into one RUN the apk add, the pip intall and the apk del to immediately remove unneeded packages.
In the past, some of our students complained that editing the source files with all the ^M (Windows) line endings in place is troublesome. These occur, depending on whom built the container where. To resolve this issue I've started including dos2unix into each Dockerfile, to clean up any scripts and other text-based files. I'm considering adding nano into the containers for those completely unfamiliar with vi.
Please let me know if you'd like me to continue these activities.