Support reproducible builds for Debian

[AkihiroSuda]

See:

• docker-library/official-images#16044

• • -

• https://github.com/reproducible-containers/repro-sources-list.sh is added to allow reconfiguring /etc/apt to use snapshot[-cloudflare].debian.org/archive/debian/<SOURCE_DATE_EPOCH>. However, this is disabled by default due to the slowness of the snapshot server. Reproducers may opt-in to enable this by specifying --secret id=enable-repro-sources-list,source=/dev/null. This is treated as a "secret" to hide it from build history for the reproducibility of the OCI config blob, but its content is not confidential.

• /var/log/* is removed as they contain timestamps

• /var/cache/ldconfig/aux-cache is removed as they contain inode number, etc.

• wget is executed with --no-hsts to disable creating /root/.wget-hsts

• To reproduce file timestamps, reproducers have to specify rewrite-timestamp=true as an exporter option (--output). This needs the master branch of BuildKit (will be included in BuildKit v0.13).

• The base image is not pinned by the hash. Reproducers may use the CONVERT action of source policies. https://github.com/moby/buildkit/blob/v0.13.0-beta1/docs/build-repro.md

[AkihiroSuda]

Result of the current revision of the PR (https://github.com/AkihiroSuda/docker-library-httpd/tree/0fffbf116b8bdb14f6969ae606766f8f9dd468d1):

$ git rev-parse HEAD
0fffbf116b8bdb14f6969ae606766f8f9dd468d1

$ sudo buildctl build --frontend dockerfile.v0 --local dockerfile=2.4 --local context=2.4 --opt build-arg:SOURCE_DATE_EPOCH=1705159178 --output type=oci,name=build-0,dest=/tmp/build-0.tar,dir=false,rewrite-timestamp=true
[...]
=> exporting to oci image format                                                                                                                                                                                                               4.0s
 => => exporting layers                                                                                                                                                                                                                         0.0s
 => => rewriting layers with source-date-epoch 1705159178 (2024-01-13 15:19:38 +0000 UTC)                                                                                                                                                       3.6s
 => => exporting manifest sha256:37daef8c7bd3c1b46d45b2fd040aa165b17afa85e8f3f845a765696785221c00                                                                                                                                               0.0s
 => => exporting config sha256:467bdfa7f53f9feef496527e163472237e7caca94d02631b074a75c802645619                                                                                                                                                 0.0s
 => => sending tarball

$ sudo buildctl build --frontend dockerfile.v0 --local dockerfile=2.4 --local context=2.4 --opt build-arg:SOURCE_DATE_EPOCH=1705159178 --output type=oci,name=build-1,dest=/tmp/build-1.tar,dir=false,rewrite-timestamp=true --no-cache --secret id=enable-repro-sources-list,source=/dev/null
[...]
=> exporting to oci image format                                                                                                                                                                                                               6.7s 
 => => exporting layers                                                                                                                                                                                                                         2.8s 
 => => rewriting layers with source-date-epoch 1705159178 (2024-01-13 15:19:38 +0000 UTC)                                                                                                                                                       3.5s 
 => => exporting manifest sha256:37daef8c7bd3c1b46d45b2fd040aa165b17afa85e8f3f845a765696785221c00                                                                                                                                               0.0s 
 => => exporting config sha256:467bdfa7f53f9feef496527e163472237e7caca94d02631b074a75c802645619                                                                                                                                                 0.0s 
 => => sending tarball

Used moby/buildkit@a09112603a36532b8e44ade74ee3fd8383f4b708 (master) on Ubuntu 23.10 amd64

[tianon]

Thank you for your contribution :heart:

As noted at https://github.com/docker-library/official-images/issues/16044#issuecomment-1894244731, this is not something we're comfortable with at this time -- perhaps at some point in the future we can revisit it.

[AkihiroSuda]

Thank you for your contribution ❤️

As noted at docker-library/official-images#16044 (comment), this is not something we're comfortable with at this time -- perhaps at some point in the future we can revisit it.

Thanks for taking a look, but there might be a confusion. This PR does not enable the slow snapshot.debian.org for the upstream builds. Using snapshot.debian.org is opt-in via --secret id=enable-repro-sources-list,source=/dev/null. This should be only opted in by third-party reproducers.

Could you consider reopening the PR if this sounds good?

[AkihiroSuda]

Alternative:

• https://github.com/docker-library/httpd/pull/250

This PR does not contain anything relevant to snapshot.debian.org. Hope this is easier to merge.