OSInside / kiwi

KIWI - Appliance Builder Next Generation
https://osinside.github.io/kiwi
GNU General Public License v3.0
298 stars 152 forks source link

support reproducible builds #2358

Open JanZerebecki opened 1 year ago

JanZerebecki commented 1 year ago

Problem description

When building the same image twice the output is different. Fix this by making the kiwi output reproducible. See https://reproducible-builds.org/ from more info on the general idea.

Expected behaviour

Output of kiwi is reproducible.

Steps to reproduce the behaviour

Branch from the Factory with: osc branch --nodevelproject openSUSE:Factory/openSUSE-Tumbleweed-Yomi Download image after it is done: osc getbinaries home:jzerebecki:branches:openSUSE:Factory/openSUSE-Tumbleweed-Yomi:livecd images/x86_64 Rebuild: osc rebuild home:jzerebecki:branches:openSUSE:Factory/openSUSE-Tumbleweed-Yomi:livecd images x86_64 Download again, build result is different.

> diff -sr binaries.kiwicd.1/openSUSE-Tumbleweed-Yomi.x86_64-1.0.0-livecd-Build21.1.iso binaries.kiwicd.2/openSUSE-Tumbleweed-Yomi.x86_64-1.0.0-livecd-Build21.2.iso
Binary files binaries.kiwicd.1/openSUSE-Tumbleweed-Yomi.x86_64-1.0.0-livecd-Build21.1.iso and binaries.kiwicd.2/openSUSE-Tumbleweed-Yomi.x86_64-1.0.0-livecd-Build21.2.iso differ

The following images show this: iso: https://build.opensuse.org/package/show/openSUSE:Factory/openSUSE-Tumbleweed-Yomi livecd

Not tested, but possibly differently affected: docker: https://build.opensuse.org/package/show/openSUSE:Factory/minimal-image vmx oem btrfs xfs vhdx vmdk qcow2: https://build.opensuse.org/package/show/openSUSE:Factory/kiwi-templates-Minimal

Perhaps some PRs I made to make the product-builder reproducible are useful: https://github.com/openSUSE/product-builder/pull/26 https://github.com/openSUSE/product-builder/pull/27 https://github.com/openSUSE/product-builder-plugins/pull/16

OS and Software information

lslezak commented 2 months ago

This not only affects the ability to verify the images but it also makes fine tuning the image size more difficult.

We build an installer Live ISO, the size of the built ISO file is around 700MB.

But the built size varies +/- 1MB, probably because some file timestamps or file order in the image can cause slightly better or worse compression.

So when I delete some not needed files from the image and the image gets 1MB smaller I do not know whether that was because of deleting those files or I was just lucky and the image was a bit more compressible and it saved 1MB.