feat: use `apt update --snapshot` for reproducibility

[endersonmaia]

This PR will change the main Dockerfile to use the apt update --snapshot option to lock the Ubuntu repository into a specific point in time.

See: https://snapshot.ubuntu.com

So, we don't need to pin package versions anymore and get the guarantee that the package versions used and their dependencies will always be the same.

When we want to change the date, we just need to change the ARG NOBLE_DATE to a desired values.

I'm reusing this same ARG to the apt update --snapshot= and for the Oficial Ubuntu Docker Image tag that has a date, like ubuntu:noble-20240801.

If we desire to change this date in the future, we can look what's the latest dated version available at https://hub.docker.com/_/ubuntu

Noble Numbat is the codename for the latest Ubuntu LTS version, which is 24.04.

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
bug-buster	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Sep 12, 2024 4:29pm

[guidanoli]

From https://snapshot.ubuntu.com/:

We intend to ensure snapshots are available for dates up to at least 2 years in the past, which we may extend if there is demand.

Should we be worried about snapshots getting deleted?

[endersonmaia]

From https://snapshot.ubuntu.com/:

We intend to ensure snapshots are available for dates up to at least 2 years in the past, which we may extend if there is demand.

Should we be worried about snapshots getting deleted?

It's 2 years, I hope we have to update something in two years.

We could create a CI check to alert us that this is getting close to the end, IDK.

[guidanoli]

It's 2 years, I hope we have to update something in two years.

But an application can (theoretically) live forever on-chain. Near this expiration date, could we at least download the snapshot locally to have it still reproducible?

[endersonmaia]

It's 2 years, I hope we have to update something in two years.

But an application can (theoretically) live forever on-chain. Near this expiration date, could we at least download the snapshot locally to have it still reproducible?

Yes, it's doable to keep a mirror ourselves.

[endersonmaia]

after my fixup at b72863b, i executed the build CI one more time to check for reproducibility

• https://github.com/crypto-bug-hunters/bug-buster/actions/runs/10834887780/job/30066431814#step:13:61
• https://github.com/crypto-bug-hunters/bug-buster/actions/runs/10834887780/job/30065136808#step:13:61

Both matches:

Cycles: 177625776
177625776: 5fe987467d7f4735f603d20a7fbf476be0b1b45229cd5b12943f50a048e7b56b

[guidanoli]

Both matches

Great! And you ran the second time without cache, right?

[endersonmaia]

Both matches

Great! And you ran the second time without cache, right?

Not sure about the cache :thinking:

I'm running locally to see if I get the same result tho.

[guidanoli]

I got a different template hash, and I've found two sources of non-determinism in the build of Lua and BusyBox.

Lua

The file permissions of bounty.lua differ. Red is mine, green is CI.

--rw-rw-r--   0        0        0     1841 1970-01-01 00:00:00.000000 bounty.lua
+-rw-r--r--   0        0        0     1841 1970-01-01 00:00:00.000000 bounty.lua

Maybe this is solved by adding a --chmod=644 option to the COPY directive in the Dockerfile?

BusyBox

It embeds the build time in its version string, which naturally differs.

-BusyBox v1.36.1.36.1 (2024-09-12 22:34:11 UTC)
+BusyBox v1.36.1.36.1 (2024-09-09 13:29:39 UTC)

One fix is to define the SOURCE_DATE_EPOCH environment variable at build time to some constant.

My local build and the CI build also differed in this NT_GNU_BUILD_ID thing, which apparently has something to do with debug symbols, but I am hopeful that, once we make this timestamp constant, this will also be solved.

[guidanoli]

Let's leave these issues to a future PR, this looks good already.