Consider squashing the image's layers

[debarshiray]

Image and version of the image where the issue happens

Possibly all the images. I have checked the images for Arch Linux and Ubuntu.

Describe the bug

The images have several layers, which are probably remnants from the lines in the Containerfiles. Compare the quay.io/toolbx-images/ubuntu-toolbox:20.04 image:

$ skopeo inspect --format '{{.Layers}}' docker://quay.io/toolbx-images/ubuntu-toolbox:20.04
[sha256:ca1778b6935686ad781c27472c4668fc61ec3aeb85494f72deb1921892b9d39e
 sha256:94e800e525a6e03e7462be3678eeebd254c96a5f6becbd9bdfe4eb494da70ec0
 sha256:dfaa34228aad276d8d6486521ae8aab7c9333673b95dbfc44a4e9b24bf149f31
 sha256:d28fd5e4448c0b2c5925d85a9021922eb93ed4c512018bb95caa75bc831b6a10
 sha256:8bd30ec7a6fe628ee9e2795a3bd2090da37bfe1261399252f441ece639dd3227
 sha256:5458ac9b664a0b144f342ef988559c3bcc21475bd6855129c13af109c95b0d16
 sha256:1e093c31394f4479ffb37dda2d044e5d0e65a3e0119378dc47aed00c6ab7889a
 sha256:b6979a5f894bbc4b25a87384a0a4c2e1ecba0fc18534d23328ad3965de0d2704
 sha256:611cc673dba45810092d9ef0ff1f2ddac64e358e62c2a952da9a625df4d2be39]

$ podman inspect --type image --format '{{json .RootFS}}' quay.io/toolbx-images/ubuntu-toolbox:20.04
{"Type":"layers",
 "Layers":
  ["sha256:6f37ca73c74f2cef0ddefd960260f2033c16c84583c5507a4f37b1cf7631dc20",
   "sha256:beda7aef5813c2426e71807aa8c39ab47ee62177d2372888da48f981dfbb9fb2",
   "sha256:d5f39395f424942be6eab2b5a3286da319cf1be909738798511373df2eba388e",
   "sha256:3f81963c31fe6def869dac5bc701e9e0f322e49ba19e4953aa010104cb40f4c0",
   "sha256:c34486955f621ac83fc2083e817cecb622963e3d5d07bff6ab9d2059ce8ec82e",
   "sha256:59845a3910aa8ff2d5b41df8ae32c7958311c3d899e8600159530ca4e5caffbd",
   "sha256:47c87d63821a98bb2c922a1bdfd190d5c5ef3d13a1a120b84e14dac97e215c9e",
   "sha256:758f372ea4d2080f2810907c5d37ba543d4462572082a77e62a444d4dd499c1e",
   "sha256:4a2dea7b1724b8eb5eca30fee9ac3d2962e2e2c91dc43491f531c8a454fd33c9"]
}

... with the registry.fedoraproject.org/fedora-toolbox:38 image:

$ skopeo inspect --format '{{.Layers}}'  docker://registry.fedoraproject.org/fedora-toolbox:38
[sha256:48031c47e5fb53ed2c92b342e03e840e1caaf263a50b9899463b1cff16574698
 sha256:69930001c1dc4e3181e33b6920f7772d0b35626a4c7ffd028a857af34c664282]

$ podman inspect --type image --format '{{json .RootFS}}' registry.fedoraproject.org/fedora-toolbox:38
{"Type":"layers",
 "Layers":
  ["sha256:a847372eb9e7cdc264e19a9b646ac0c0ed8cc870b124efd34d1241b9dd47d5c9",
   "sha256:1c4aad3e00bcfccfc8ede1e0f65690e21ea6610b25c5cecb87a40467aadb637a"]
}

The Fedora build system squashes the layers. It uses Docker for the build, but I think it's the equivalent of podman build --squash. The other option is to hack the Containerfiles in a way that it avoids the creation of extra layers, but that might eventually lead to convulated code.

Reproduction steps

See above.

Host distribution and version, toolbx and podman versions

podman-4.4.1-3.fc36.x86_64

[Jmennius]

Just a note - with Ubuntu images I've made sure not to blow up layers in size - so all size-sensitive operations are done in a single command. I wonder if we should use --squash or --squash-all (I believe only the latter will provide additional size benefits).

[debarshiray]

Some historical context: https://src.fedoraproject.org/container/fedora-toolbox/pull-request/4#comment-132348

Just a note - with Ubuntu images I've made sure not to blow up layers in size - so all size-sensitive operations are done in a single command.

Cool!

I wonder if we should use --squash or --squash-all (I believe only the latter will provide additional size benefits).

The Fedora build system uses Docker to build images, while I have always used podman build myself. So, I am not sure if what Fedora is doing is equivalent to --squash or --squash-all. However, the podman-build(1) manual suggests that it's --squash.

I am no expert in Container/Dockerfiles, but I have observed that it's considered good practice to reduce the number of layers in the images. Sometimes that's done by cleverly hacking the Container/Dockerfile. I am worried that with a sufficiently complex Container/Dockerfile these hacks will lead to convoluted code that's hard to read and debug.

Therefore, I was thinking that it might be simpler to have the build doing the squashing for us.

[debarshiray]

Just an update:

From Fedora 39 onwards, the fedora-toolbox image is no longer layered and is built without any layers as another base image. They are now part of the nightly Fedora composes and no longer written with Container/Dockerfiles.

[debarshiray]

I suspect that the switch to Podman and Buildah based GitHub Actions to build the images also fixed this. Compare this to the earlier output above:

$ skopeo inspect --format '{{.Layers}}' docker://quay.io/toolbx-images/ubuntu-toolbox:20.04
[sha256:fa1c8595265700fcd3c36de73ee8167969927e8661c2168508f18857f275f752
 sha256:88968c9355b780e3702af2ce6a374f6d641476380b294e9b97a68de1e989f9b2]

I see that we are using layers: false with redhat-actions/buildah-build@v2. I am not sure exactly what it means, and there are subtle differences between podman build --squash and buildah bud --squash (it's the same as podman build --squash-all). I suppose, we can use extra-args: ... to clarify this further, but looking at the skopeo(1) output above, I think this is fixed.

[travier]

Thanks for the update!