trusting substituters, but not *their* subsituters (non-transitively)

[mschwaig]

Is your feature request related to a problem? Please describe.

As a user, who trusts a given substituter, I can add its public key to trusted-public-keys inside nix.confand substitute any paths it already built from there. The detached signatures that make this possible already link a set of inputs with the corresponding output produced by the build step (https://discourse.nixos.org/t/what-guarantees-do-signatures-by-binary-caches-give/34802/3). Such signatures could also link each of these input output pairs to the specific builder that produced the output, but this is currently only done on a human level by knowing how the involved hosts are configured. Working only with this implicit knowledge is possible in practice but not well suited for reproducibility tracking or other supply chain security purposes.

The main problem is that there is no way for us to tell how the substituter got the output. It might have

1. built the output itself,
2. obtained the output from another builder or substituter that it trusts or
3. (as far as I can tell) the output might have also been added manually to the served store, without any traceable evidence about how it was produced.

This gap is especially bad for reproducibility tracking, because (2) looks exactly like (1) if the derivation in question was reproducible.

Describe the solution you'd like

The detached signatures obtained from substitutes should optionally also cover the extra information that "The signer has built this output itself." (and unmodified Nix binaries should enforce that they cannot lie about this).

A user should be able to configure Nix so that substitution only succeeds if the builder can be identified and is trusted.

When the builder and substituter are the same this clearly solves the problem. If the builder and substituer are different users need to configure/manage additional signing keys, and the substituter would need to appropriately forward along the signature of the builder. I have not verified if this is possible in Nix already. EDIT: nix-serve can return multiple signatures already so it looks like that's supported.

Note that I expect this feature would not resist tampering (with modified versions of Nix, otherwise modified build hosts or a malicious dependency of the builder compromising the build sandbox), but in contrast to trusted-public-keys as-is non-malicious users should never stumble into bypassing it.

Describe alternatives you've considered

I am aware of Trustix, but perfect is the enemy of good, therefore I think any improvements to attribution of build steps to builders that are possible with the existing signing scheme should also be considered as a constructive addition.

Additional context

I'm currently doing research on these issues which I would like to continue and am looking for funding and collaborators.

Priorities

Add :+1: to issues you find important.

[mschwaig]

There is already an issue in Nix on creating signatures in the default configuration (https://github.com/NixOS/nix/issues/3023), which also helps with attribution, but still requires some implicit knowledge about the configuration as far as I can tell.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/what-guarantees-do-signatures-by-binary-caches-give/34802/4

[mschwaig]

I took a closer look at how this could be implemented.

The approach that I came up with is a new version of ValidPathInfo::fingerprint, which is the data that gets signed.

In the current version it looks like this

1;<store path>;<NAR hash in base32 including its own type>;<NAR size in bytes>;<NAR references separated by ,>

while I would propose to adding an origin info field in a newer revision

2;<store path><NAR hash in base32 including its own type>;<NAR size in bytes>;<origin info>;<NAR references separated by ,>

This field would be a string for extensibility, but there are already some good candidates for how origin information could be added.

"": In case we don´t know how that path got into the store/none of the other cases apply. "database": In case the Nix database indicates that the host built this path itself. Can be determined, and therefore generated, on the fly. "builder-sig": In case the signature was generated by the builder right away.

The required info for the database value is already available in the result of nix path-info --json in form of the ultimate field being set to true ONLY for paths built by a given host and recorded all the way back since Nix 2.0. For example:

$ nix path-info .# --json | jq
[
  {
    "deriver": "/nix/store/6liymdkgg11sc71rfym3qipd1i86mvyf-latex-document.drv",
    "narHash": "sha256-O52rerUSCYC4z3Ey4YJzh4jIC6MHEkkr5UtIL6fkC+E=",
    "narSize": 1822896,
    "path": "/nix/store/0w7s3x0x28gl3i81jgcdf7xvpv0x250b-latex-document",
    "references": [],
    "registrationTime": 1704379455,
    "ultimate": true,
    "valid": true
  }
]

The builder-sig value could be used whenever locally-built paths are signed automatically (and a specific config flag is set).

In general I would assume that substituters would opt-into additionally providing the new signature format on a second signature line for backward compatibility with clients that don´t know about the new format.

(While we're at it other additions to the fingerprint data would be possible, for example a rebuild count and the registration time. I might file a follow-up issue for why replacing the ultimate field with a rebuild counter would be great.)

[mschwaig]

Since the origin info is specific to each signature and in the long term guessing different origin info values during validation would be really ugly the signature format would also need to be adapted from

<name;<signature>

to something that includes the origin data for inclusion in the fingerprint, like

<name>?v=2&origin=<origin info>;<signature>

which would hopefully at the same time just make older clients disregard the signature as invalid.

Funnily enough with this you could also make existing clients only accept paths that a specific host built itself and (claims to have) signed right away with a config like

trusted-public-keys = [ "my.buildhost.net?origin=builder-sig:e9ZHfXR0uwMtkhjIsjjsjBBD38WKUShh3hBtSkURcL6JoBQ==" ];

The nix DB should be happy with all of this as well since signatures are just a text field with space-separated signatures.

[mschwaig]

@lheckemann brought to my attention that it would make sense to add a content hash of each (direct) build dependency to the fingerprint as well. Otherwise you have the same trust issue with the build dependencies that can for sure affect the built output, and no way to verify those.

I have not yet managed to figure out if any addition is required to make that possible for CA derivations:

About derivations I am wondering if the References inside .narinfo files are content hashes. About realisations I am wondering if the dependentRealisations field boils down to a content hash.

In the end it would make sense to me if the produced signatures made it possible for a verifying party to understand how the builder resolved individual dependencies, either during realisation or potentially also for non CA-derivations.

[roberth]

About derivations I am wondering if the References inside .narinfo files are content hashes.

They are essentially store paths, which may be input addressed, and therefore not content hashes.

About realisations I am wondering if the dependentRealisations field boils down to a content hash.

"dependent" suggests that it's about the reverse relation, ie potentially the whole store.

But tbh, I have no idea and CA is completely undocumented.

The glossary does not acknowledge the existence of realisations: https://nixos.org/manual/nix/stable/glossary.html?highlight=realisation#glossary

The page about nix realisation contains 0 information. An AI could generate it without knowing what Nix is. https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-realisation.html?highlight=realisation#synopsis

https://nixos.org/manual/nix/stable/command-ref/new-cli/nix3-realisation-info is hardly any better.

contentAddressed does not mention it either https://nixos.org/manual/nix/stable/language/advanced-attributes#adv-attr-contentAddressed

cc @thufschmitt

[emilazy]

A machine that trusts an untrustworthy substituter cannot be trusted to be a substituter, because any trusted untrustworthy substituter can easily backdoor the entire machine and cause it to build more untrustworthy outputs. As a result, I don’t think this information would give any benefit or additional guarantees; it would be trivially circumventable by a malicious substituter and offer a false sense of security.

[mschwaig]

@emilazy I looked into this more since my last comment, and it's a nuanced topic. The ability for the signer to link additional data to the signature, by including it in the signature, can in fact lead to better security guarantees.

I actually wrote a paper about what is possible and how, which I am currently polishing for publication. You can find the draft here: https://discourse.nixos.org/t/extending-cloud-build-systems-to-make-transitive-trust-explicit/50841

I cannot give a detailed technical answer here right now, because I am still quite busy with finishing the paper, but I do want to communicate the ideas from that paper to the community, once I have the time to do it properly and figure out what to put where.

You and anyone is welcome to take a look if your interested and give feedback in that discourse thread, or contact me directly, for example on Matrix with more specific questions.

[emilazy]

Thanks for the link, and congratulations on the paper! I’ll have to give it a read. I know that there are many root‐level exploits you can do with Nix if you are a trusted substituter or a trusted user, but perhaps the picture is more nuanced than I think.