grahamc
commented
5 years ago The current architecture is very simple:
Given an input expression:
We evaluate the derivation:
and then query all of the dependencies of that derivation:
Each dependency gets added to a queue for verification:
Then, for each dependency we build it twice. First with a fairly standard set of build options, where it will almost certainly be fetched from the binary cache:
Then we build it a second time with --check, forcing the local machine to build it again:
Nix will then exit 0 if the output matched bit-for-bit the original build: https://github.com/grahamc/r13y.com/blob/f144539ae80108bd8d7bf243d67011ca63198dce/src/bin/check.rs#L247
Otherwise the build is not reproducible, and we do some fancier things:
Notably we take the original build output, and the not-matching build output and export them as a NAR, and then copy them to a content-addressed store (CAS):
The result of this process is a JSON document containing a list of builds: an object saying the path was reproducible, or an object which includes the derivation and the hash of the two NARs.
A separate process, report.rs starts a similar way. Given an expression:
Evaluate it and find a list of derivations and dependencies (todo: refactor :) ):
and create a report of all the paths:
Given an unreproducible path, it takes the two NARs from the CAS, extracts them to the Nix store and runs Diffoscope on them:
From there, a simple HTML report is generated:
The ideas behind using this instruction, the JSON document, and the CAS store is explicitly designed around being able to distribute the work to many builders, to make it easy to grow this project later. I would love help with this. You can see the data types I thought about here, which are designed with this in mind: https://github.com/grahamc/r13y.com/blob/f144539ae80108bd8d7bf243d67011ca63198dce/src/messages.rs
One thing these data types assume is that every builder would randomize the list of derivations and try to build all of them. The idea being have many builders try the same thing makes us more sure about the reproducibility. Now I wonder if we would want something a bit different, to allow greater coverage. My thinking now is the central server would publish the same instruction, but also publish statistics about how many times each derivation has been built. In this way builders can prioritize a low-count derivations first.
Does this help?
davidak
This project is amazing! It shows that we are very far in the goal to make nixos reproducible.
At least for _nixos-unstable's iso_minimal job for x8664-linux.
It would be nice to have this metric for the stable releases. We might get some headlines for nixos and reproducibility.
It would also be nice to extend the scope of this "experiment" and integrate it into nixos. I'm not sure how this works because there is no documentation (https://github.com/grahamc/r13y.com/issues/5), but you probably compare hashes? So, hydra should calculate this hashes and save them in the metadata of all builds. Then we need some distributed computing infrastructure where the community can run builds on their machines and submit their hashes. Then a central instance can calculate how reproducible each package is that has this data available. We can use that data on our website and package search.
I know BOINC for distributed computing tasks, but we might find a less complex solution.
I would like to just execute one command to build packages from a job set or channel i like and submit hashes.