Glossary: reproducible

[davidak]

https://nix.dev/glossary currently says:

Reproducibility would guarantee exactly the same results no matter when or on what machine you run the command.

I think this is not clear to everyone and maybe not actually what we mean when talking about nix and nixos. It might be true in the narrow context of software developers that work for a company that only use x86_64-linux systems, but the real world is more complex.

Since that page became authoritative terminology (with https://github.com/NixOS/nix.dev/issues/377) for the full project including documentation and i guess also public communication, we should review it to the same standard as the new definitions!

First, what does result mean? Is it the binary package, the terminal output of a command or the function of a program?

In the context of the reproducible builds initiative, it means having bit-by-bit identical binary artifacts when building a package.

Since we use the same term on https://nixos.org/, i guess we agree on their definition.

https://reproducible-builds.org/docs/definition/

But this means, building a package for different architectures, you get a different artifact! So the program might also work differently.

So it is in conflict with our current definition here.

no matter [...] on what machine you run the command

We have an example here where the same definition is used:

https://github.com/NixOS/nix.dev/blob/master/source/tutorials/ad-hoc-developer-environments.md#towards-reproducibility

In that case, it matters on what machine i run the command!

On NixOS it will work, on macOS it might work, on a Linux on RISC-V it might not work and on Windows, it will not work (since Nix does not support it except through WSL).

I also miss that detail from the reproducible builds initiative definition. There are more systems than x86_64-linux! In the software industry, most developers probably don't use Linux! And with computer users even less.

So we can't really claim:

if a package works on one machine, it will also work on another

which is the core claim of the whole project and on https://nixos.org/.

People that look at the project for the first time and understand it often think "this sounds too good to be true" and i think we should be very honest about the limitations of our solution to not look like a scam.

What does "the command" mean here? The nix command or like in the linked example, git? Can Nix really make every program reproducible? If i run a game, do i get the same score every time i play? (no, many programs don't function that way)

What do we really mean when we say reproducible?

I think a good metric would be that high level programmers like JavaScript and Ubuntu users should understand it.

We could also add more words, like declarative and reliable, that are prominently used on the website.

cc @fricklerhandwerk

[fricklerhandwerk]

Fully agree with the problem statement. A term people tend to use is "repeatable", which seems to focus more on the outcome of computations or program behavior. It would be great if you could make a proposal in form of a PR so we can spread this through the docs team and get the right people involved who may have relevant opinions on this.

[davidak]

Another term is "composeable" environments.

But the marketing team decided to make this the header of the website: "Reproducible builds and deployments.". So we might stick to these two terms and define them properly.

For "reproducible builds" we should stick to the well-known meaning and linking to the definition of the initiative.

I created a PR to remove the definition for now to not cause any more damage. I have to add that that one term is not part of the official glossery whenever i link to that page, which enforces the image that this project is a complete mess. We can't even find words to describe what we are even doing here :(

[fricklerhandwerk]

Referring to https://reproducible-builds.org is a good idea. We may need to reword a lot of stuff though, so for now it's fine to leave the term out of the glossary until we find time to take care of that.

the image that this project is a complete mess

In a sense it is, as it has grown without much direction or coordination for many years. It's frustrating and there is no one to blame. We're working on it right here right now.

We can't even find words to describe what we are even doing here :(

Yes, that's in fact one of the hardest parts, because it's the basis for our mindshare.