heroku / cnb-shim

This is a shim to run old buildpacks as Cloud Native Buildpacks
MIT License
1 stars 0 forks source link

Ensure generated archives are deterministic #45

Closed edmorley closed 2 years ago

edmorley commented 2 years ago

Previously each request to cnb-shim for the same buildpack would result in a tarfile with varying contents even if the underlying buildpack hadn't changed.

This was due to:

With the changes here, repeat requests now result in an identical archive, so long as the buildpack or shim scripts haven't changed.

This helps reduce layer churn in builder images containing shimmed buildpacks, such as our new heroku/builder-classic:22 builder, or the older heroku/buildpacks:{18,22} builders.

The new tar arguments are based on those recommended here: https://reproducible-builds.org/docs/archives/

GUS-W-11277378.

edmorley commented 2 years ago

Example differences (prior to this PR) between archives generated from separate requests to https://cnb-shim.herokuapp.com/v1/heroku/python?version=0.0.0&name=Python:

diffoscope screenshot

(comparison generated using https://diffoscope.org)

edmorley commented 2 years ago

This + #46 deployed to production, and a heroku/builder CI run kicked off to confirm all still working: https://app.circleci.com/pipelines/github/heroku/builder/1885/workflows/727394fd-f999-4cbc-aa2e-f014ea521c93

(I'd already tested these changes briefly prior to opening the PRs, but not to the extent that the builder CI run will)

edmorley commented 2 years ago

Also:

$ curl -sSf 'https://cnb-shim.herokuapp.com/v1/heroku/python?version=0.0.0&name=Python' -o shimmed-python1.tar.gz
$ curl -sSf 'https://cnb-shim.herokuapp.com/v1/heroku/python?version=0.0.0&name=Python' -o shimmed-python2.tar.gz
$ sha256sum shimmed-python*
0d7fb01139bc02e9ab26c7d443a5b1c7eafa88b662e8518626e3b1c41e56e5d7  shimmed-python1.tar.gz
0d7fb01139bc02e9ab26c7d443a5b1c7eafa88b662e8518626e3b1c41e56e5d7  shimmed-python2.tar.gz