Support reproducible builds using SOURCE_DATE_EPOCH

nicholasdille commented 8 months ago

Description

Right now, builds are not reproducible. An important step towards that goal would be supporting timestamps as per reproducible-builds.org and the Docker blog.

I propose a new field called source-date-epoch with the following behaviour:

If unset, builds are performed as before for backwards compatibility
If set, the env var SOURCE_DATE_EPOCH is set to the specified value and is passed on to docker

crazy-max commented 8 months ago

It should already be possible using the env property:

      -
        name: Build and push
        uses: docker/build-push-action@v5
        with:
          context: .
          push: true
          tags: user/app:latest
        env:
          SOURCE_DATE_EPOCH: 0

@dvdksn Maybe we could have a new section in https://docs.docker.com/build/ci/github-actions/ about reproducible builds?

dvdksn commented 8 months ago

Yeah sounds like a good addition. Noted

huxuan commented 3 months ago

Repost from https://github.com/docker/build-push-action/issues/1043

Contributing guidelines

[X] I've read the contributing guidelines and wholeheartedly agree

I've found a bug, and:

[X] The documentation does not mention anything about my problem
[X] There are no open or closed issues that are related to my problem

Description

The logs show that the sha256 for all layers for the registry cache are the same except for config.

Expected behaviour

The sha256 for the registry cache config should be the same.

Actual behaviour

The sha256 for the registry cache config are mismatched which leads to untaged packages.

Repository URL

https://github.com/huxuan/ss-python

Workflow run URL

/actions/runs/7709174339/job/21009975797

YAML workflow

container-publish:
    needs: release-publish
    permissions:
      contents: read
      packages: write
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0
      - uses: docker/setup-buildx-action@v3
      - uses: docker/login-action@v3
        with:
          password: ${{ secrets.GITHUB_TOKEN }}
          registry: ghcr.io
          username: ${{ github.actor }}
      - env:
          SOURCE_DATE_EPOCH: 0
        uses: docker/build-push-action@v5
        with:
          build-args: |
            PYTHON_VERSION=${{ matrix.python-version }}
          cache-from: |
            type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-dev-py${{ matrix.python-version }}
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-dev-py${{ matrix.python-version }},mode=max
          context: .
          file: .devcontainer/prebuild/.devcontainer/Dockerfile
          provenance: false
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:dev-py${{ matrix.python-version }}
            ghcr.io/${{ github.repository }}:dev-py${{ matrix.python-version }}-${{ github.ref_name }}
          target: dev
      - env:
          SOURCE_DATE_EPOCH: 0
        uses: docker/build-push-action@v5
        with:
          build-args: |
            PYTHON_VERSION=${{ matrix.python-version }}
            PDM_BUILD_SCM_VERSION=${{ github.ref_name }}
          cache-from: |
            type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-dev-py${{ matrix.python-version }}
            type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-prod-py${{ matrix.python-version }}
          cache-to: type=registry,ref=ghcr.io/${{ github.repository }}:build-cache-prod-py${{ matrix.python-version }},mode=max
          context: .
          file: .devcontainer/prebuild/.devcontainer/Dockerfile
          provenance: false
          push: true
          tags: |
            ghcr.io/${{ github.repository }}:prod-py${{ matrix.python-version }}
            ghcr.io/${{ github.repository }}:prod-py${{ matrix.python-version }}-${{ github.ref_name }}
          target: prod
    strategy:
      matrix:
        python-version:
          - '3.8'
          - '3.9'
          - '3.10'
          - '3.11'
          - '3.12'

Workflow logs

Attempt 1: https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395

#22 exporting cache to registry
#22 preparing build cache for export
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173[377](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:382)497d6e
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173377497d6e 0.2s done
#22 writing layer sha256:1b13d4e1a46e5e969702ec92b7c787c1b6891bff7c21ad[378](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:383)ff6dbc9e751d5d4
#22 writing layer sha256:1b13d4e1a46e5e969702ec92b7c787c1b6891bff7c21ad378ff6dbc9e751d5d4 0.1s done
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c 0.1s done
#22 writing layer sha256:20f18e486fc01f87b6b311be20ae9db9cb82df60c14ad1783b86d5fa2ba623f8
#22 writing layer sha256:20f18e486fc01f87b6b311be20ae9db9cb82df60c14ad1783b86d5fa2ba623f8 0.2s done
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c 0.2s done
#22 writing layer sha256:2878b437a07192a6cf2b2075786d0a7922b16f1cee9dad8a5f305deef52e6253
#22 writing layer sha256:2878b437a07192a6cf2b2075786d0a7922b16f1cee9dad8a5f305deef52e6253 1.3s done
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1 0.1s done
#22 writing layer sha256:30d85599795460b2d9d24c6b87c53ec60555b601705cc83bea31632240500980
#22 writing layer sha256:30d85599795460b2d9d24c6b87c53ec60555b601705cc83bea31632240500980 0.1s done
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f 0.1s done
#22 writing layer sha256:396f0fee52796a6a98fc294805b23c509beb8cd0b4556e5568b4298a0be6e3c8
#22 writing layer sha256:396f0fee52796a6a98fc294805b23c509beb8cd0b4556e5568b4298a0be6e3c8 0.1s done
#22 writing layer sha256:3f00b3697662aa214d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83
#22 writing layer sha256:3f00b3697662aa214d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83 0.2s done
#22 writing layer sha256:4acda0384fa4980f4c24482858b8894f5711c36ca9269d50fbbd88bbf74de8a4
#22 writing layer sha256:4acda0384fa4980f4c24482858b8894f5711c36ca9269d50fbbd88bbf74de8a4 0.1s done
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0.1s done
#22 writing layer sha256:69c6a95e6fa75b2a15dcc1d93e96f2b16e96be599a4c74574ab1c63126e25f97
#22 writing layer sha256:69c6a95e6fa75b2a15dcc1d93e96f2b16e96be599a4c74574ab1c63126e25f97 0.1s done
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a 0.1s done
#22 writing layer sha256:76530ff9b004d5b0ad94cc95e54ad2749c0deb9d3d105d15b84049b006f9abed
#22 writing layer sha256:76530ff9b004d5b0ad94cc95e54ad2749c0deb9d3d105d15b84049b006f9abed 0.1s done
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72 0.1s done
#22 writing layer sha256:8ea4555671ddb9b06b855c89880e2946201b5ced31dbf1ce8f5b872b55b224e7
#22 writing layer sha256:8ea4555671ddb9b06b855c89880e2946201b5ced31dbf1ce8f5b872b55b224e7 0.1s done
#22 writing layer sha256:ad5739181616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a[379](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:384)b3a2
#22 writing layer sha256:ad57[391](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:396)81616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a379b3a2 0.1s done
#22 writing layer sha256:b3ba329ce47347173b038e63b8c9a78f7cf20324c2377dc3a0904837cfbe2e23
#22 writing layer sha256:b3ba329ce47347173b038e63b8c9a78f7cf20324c2377dc3a0904837cfbe2e23 0.1s done
#22 writing layer sha256:c12af4d99b224c7583539128a04c9[395](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009847395#step:6:400)7b1908684e50013d8b58a8e2988d7b9f
#22 writing layer sha256:c12af4d99b224c7583539128a04c93957b1908684e50013d8b58a8e2988d7b9f 0.1s done
#22 writing layer sha256:f4cb18646a15052fbff9e6a7f2af27b73ce3033e9e973709fedf06058817f092
#22 writing layer sha256:f4cb18646a15052fbff9e6a7f2af27b73ce3033e9e973709fedf06058817f092 0.1s done
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2 0.1s done
#22 writing layer sha256:ffaf3b259c1baad7a193f840f1f0506f9debd25f0ba8d9de0033a61553d0f53d
#22 writing layer sha256:ffaf3b259c1baad7a193f840f1f0506f9debd25f0ba8d9de0033a61553d0f53d 0.1s done
#22 writing config sha256:20bd3b4cbe80bae9a81f88dade1f0f6a84ff8ae28a22ceaeec893b4ad92d21a6
#22 writing config sha256:20bd3b4cbe80bae9a81f88dade1f0f6a84ff8ae28a22ceaeec893b4ad92d21a6 0.9s done
#22 writing cache manifest sha256:11dd2cfbbd2816313b0fe862ee8c8cead80984e3d1a8e7b274e6197877ae815b
#22 preparing build cache for export 6.8s done
#22 writing cache manifest sha256:11dd2cfbbd2816313b0fe862ee8c8cead80984e3d1a8e7b274e6197877ae815b 0.7s done
#22 DONE 6.8s

Attempt 2: https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797

#22 exporting cache to registry
#22 preparing build cache for export
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173377497d6e
#22 writing layer sha256:0a329b671abfc277cf19a2670d019384c564e146c9c5ed1bb97d173377497d6e 0.1s done
#22 writing layer sha256:1b13d4e1a46e5e969702ec92b7c787c1b6891bff7c21ad378ff6dbc9e751d5d4 0.1s done
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c
#22 writing layer sha256:1c74526957fc2157e8b0989072dc99b9582b398c12d1dcd40270fd76231bab0c 0.1s done
#22 writing layer sha256:20f18e486fc01f87b6b311be20ae9db9cb82df60c14ad1783b86d5fa2ba623f8 0.1s done
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c
#22 writing layer sha256:26916576c92c435f7441cb7490dd537040d28b42ab2719ea02ba909f0985a57c 0.1s done
#22 writing layer sha256:2878b437a07192a6cf2b2075786d0a7922b16f1cee9dad8a5f305deef52e6253 0.1s done
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1
#22 writing layer sha256:2f44b7a888fa005d07c031d3cfad2a1c0344207def2ab9dbb97712425ff812c1 0.1s done
#22 writing layer sha256:30d85599795460b2d9d24c6b87c53ec60555b601705cc83bea31632240500980 0.0s done
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f
#22 writing layer sha256:382cd2159c71b198be5533c09c741cdfecf7dc5618feb5637bbe5f036f73d89f 0.1s done
#22 writing layer sha256:396f0fee52796a6a98fc294805b23c509beb8cd0b4556e5568b4298a0be6e3c8 0.1s done
#22 writing layer sha256:3f00b3697662aa[214](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797#step:6:219)d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83
#22 writing layer sha256:3f00b3697662aa214d22bb76bbbf5fa4d80f7ba9ca9f9076a9440d53bf529b83 0.1s done
#22 writing layer sha256:4acda0384fa4980f4c24482858b8894f5711c36ca9269d50fbbd88bbf74de8a4 0.0s done
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1
#22 writing layer sha256:4f4fb700ef54461cfa02571ae0db9a0dc1e0cdb5577484a6d75e68dc38e8acc1 0.1s done
#22 writing layer sha256:69c6a95e6fa75b2a15dcc1d93e96f2b16e96be599a4c74574ab1c63126e25f97 0.1s done
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a
#22 writing layer sha256:75e2b45cbee50cea4b3ed4f4fe167ad5994622d77a54adde89adcfeefa3c229a 0.1s done
#22 writing layer sha256:76530ff9b004d5b0ad94cc95e54ad2749c0deb9d3d105d15b84049b006f9abed 0.1s done
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72
#22 writing layer sha256:76c111f84668456d1a37f894e2a1a6fc655cecf002d8b3f7934f4ffc14231f72 0.1s done
#22 writing layer sha256:8ea4555671ddb9b06b855c89880e2946201b5ced31dbf1ce8f5b872b55b224e7 0.1s done
#22 writing layer sha256:ad5739181616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a379b3a2
#22 writing layer sha256:ad5739181616b815fae7edc6bba689496674acbcf44e48a57fc7cc13a379b3a2 0.1s done
#22 writing layer sha256:b3ba329ce47347173b038e63b8c9a78f7cf20324c2377dc3a0904837cfbe2e23 0.1s done
#22 writing layer sha256:c12af4d99b224c7583539128a04c93957b1908684e50013d8b58a8e2988d7b9f
#22 writing layer sha256:c12af4d99b224c7583539128a04c93957b1908684e50013d8b58a8e2988d7b9f 0.1s done
#22 writing layer sha256:f4cb18646a15052fbff9e6a7f2af27b73ce3033e9e973709fedf06058817f092 0.1s done
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2
#22 writing layer sha256:fa8b5ed51b617f793487eb8b5c0d947553c6d649eabd5615e35795e71c70feb2 0.1s done
#22 writing layer sha256:ffaf3b259c1baad7a193f840f1f0506f9debd25f0ba8d9de0033a61553d0f53d 0.1s done
#22 writing config sha256:336be0cc3b08f2c94c32c45c4234ae9075cb071431e19c4787dc7e51e1a5b54d
#22 writing config sha256:336be0cc3b08f2c94c32c45c4234ae9075cb071431e19c4787dc7e51e1a5b54d 0.8s done
#22 writing cache manifest sha256:fdd22d7352e00b75d70666789bd229fa24f5afebb9a[220](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797#step:6:225)def5db47f648d3f50c
#22 preparing build cache for export 3.6s done
#22 writing cache manifest sha256:fdd22d7352e00b75d70666789bd[229](https://github.com/huxuan/ss-python/actions/runs/7709174339/job/21009975797#step:6:234)fa24f5afebb9a220def5db47f648d3f50c 0.9s done
#22 DONE 3.6s

BuildKit logs

No response

Additional info

To make everything reproducible as much as possible, both
```
env:
 SOURCE_DATE_EPOCH: 0
```
and
```
with:
 provenance: false
```
are set according to https://github.com/docker/build-push-action/issues/994 and https://github.com/docker/build-push-action/issues/894
The sha256 of layers and config for single-stage are the same both for the resulting image and registry cache.
The sha256 of layers and config for multi-stage are the same only for the resulting image.
The cache overall works as expected, the only side effects are untaged packages for the registry caches.

docker / build-push-action