[FeatureRequest] Move squash out of experimental

[dhirschfeld]

...and implement for all platforms (xref: #31468)

[dhirschfeld]

There is a long discussion in #34565 which shows many valid usecases for squash which can't be duplicated using multi-stage builds.

squash is complimentary to multi-stage builds and adds significant value in situations where multi-stage builds can't be used.

[dhirschfeld]

xref: https://github.com/moby/moby/issues/33581#issuecomment-324519139

[esomir]

I completely agree! We need squash in combination with multi-stage builds, and it's a showstopper that we have to enable experimental mode for a feature that shouldn't be experimental anymore!

[williamoverton]

Would be great to have this! I work with some very large docker images due to multistage builds.

[cpuguy83]

Why would multistage builds create large images? People often use multistage to reduce size without requiring --squash

[williamoverton]

@cpuguy83 They are managed by different teams have include multiple sets of apt-get update (about 30mb) etc in the different layers. We have our linux image which is used by our nodejs image which is used by our app images etc.

If we could squash at the end that would be the solution for us, sadly we can not use the experimental build of docker.

[Heatmanofurioso]

+1

[veritas9872]

Hello. Has there been any progress on this issue?

I currently have to change the ownership of a large folder, which significantly increases image size. However, I cannot how to do this efficiently with multi-stage builds as mentioned in https://github.com/moby/moby/issues/34565.

I believe that having --squash would be much more intuitive and easier to understand than using multi-stage builds for everything.

Considering that this functionality has been in experimental for years, I would like to ask if anyone is willing to move it out of experimental. Many people who have to deal with non-root users would be very pleased with this change.

[coryvirok]

FWIW - The official ubuntu:xenial image contains ~50MB of unused layer data in /var/lib/apt/lists/. In my images where I use this as the base image, there's no other way to remove the duplicate data, (that I know of) without using --squash on my image, which is typically built by a multi-stage Dockerfile.

So, 👍🏻 to --squash being necessary even with multi-stage builds.

[thaJeztah]

FWIW - The official ubuntu:xenial image contains ~50MB of unused layer data in /var/lib/apt/lists/. In my images where I use this as the base image, there's no other way to remove the duplicate data, (that I know of) without using --squash on my image, which is typically built by a multi-stage Dockerfile.

Note that that specific issue is only present in Ubuntu Xenial images; the rootfs tarball for those images contains those files, but later versions of the ubuntu image (18.04 and up); no longer have this, and only have a single layer; see https://github.com/tianon/docker-brew-ubuntu-core/blob/b5a2c2a86e9dc83ef3a21551ea97e351697ed2d7/update.sh#L69-L76, and https://bugs.launchpad.net/cloud-images/+bug/1699913. Ubuntu Xenial reached EOL, so I would highly recommend updating your images to use a still maintained version of Ubuntu.

I should add that, if you have multiple images using ubuntu:xenial as parent image, squashing your images will result in more data to be downloaded; while it's not ideal that the ubuntu:xenial image contains that data, those layers will be shared between all images based on it. Squashing those layers will result in those shared layers to be squashed into "unique" layers for each of those images, and no longer shared.

[Morriz]

bump

[dhirschfeld]

I should add that, if you have multiple images using ubuntu:xenial as parent image, squashing your images will result in more data to be downloaded

I think by default squash should only squash up to, but not including the base image so that the final image would have only 2 layers - one being the base image and the other, everything you added in the build. Ideally, you could choose which layers you wanted to squash.

[shinsenter]

Hi all, I made a small script to combine all docker image layers into one, and also delete junk files in the image. This script is an alternative to the still experimental squash flag in docker build. https://gist.github.com/shinsenter/4f1f4f16a1f5b9a02ef9ba4faf19233d