Credentials provider support for builtins.fetch*

[simonzkl]

Is your feature request related to a problem? Please describe.

We desperately need some sort of credentials provider support for builtins.fetch* functions. The best you can currently do is to use pkgs.fetchurl, expose your credentials in plain text at some globally accessible path like /etc/nix/my-creds and add it to extra-sandbox-paths. You can restrict access to only nixbld though it's not like it matters because anyone who has access to the nix builder can echo the credentials and fetch them from the build log.

Describe the solution you'd like

• Similar to fetchGit which is able to use your local ssh agent, you should have some form of way to securely fetch packages on the client side by authenticating locally using a custom credential provider.
• Packages behind authentication would be fetched on the client side and then passed to the builder.
• Credential provider could simply read from a local file or execute a custom command (e.g. fetch credentials from pass or the system keychain).
• Credential provider would be able to set any http header for fetchurl.
• Credentials should not be saved to the nix store as that is not secure. Credentials can be expiring and user-specific so they're not reproducible anyway.

Describe alternatives you've considered

• [RFC 0143] Nix Store ACLs is not a solution to this because credentials can be user-specific and expiring which makes them non-reproducible.
• access-tokens currently doesn't work, is limited to credentials being exposed in plain text, assumes the credentials don't need to be refreshed and is limited to oauth/pat for specific platforms like gitlab/github.

Somewhat related

• Expected AWS S3 Credential Locations are Inconsistent Across nix Commands for Daemon Users

Priorities

Add :+1: to issues you find important.

[yes91]

This is a major blocker for adopting nix at my organization. I've made a recent push to use nix for our internal projects, but the only thing standing in the way is that we are hosting some binary inputs on a private Artifactory instance. There are multiple authentication methods but they all require setting http headers for fetchurl. I have sketched a solution that extends the tarball fetcher with support for access-tokens, but this feels like a hack and is far from ideal for the reasons described above. Since flakes depend on libfetchers, I feel this is also an extremely important feature for stabilizing flakes.

[ck3mp3r]

This is also blocking our internal adoption as we cannot write derivations that can download github release assets, fetchurl actually just downloads the SSO page and thinks that is the requested target.

I am not sure how far --access-tokens and --extra-access-tokens go down the chain... one idea I had was to implement a custom fetcher using the gh cli, but somehow I need to pass credentials down to the gh cli... it would be great if the access-tokens where available in the one or more of the phases... and as we use flakes arg and argstr are not an option.

[szlend]

I am not sure how far --access-tokens and --extra-access-tokens go down the chain... one idea I had was to implement a custom fetcher using the gh cli, but somehow I need to pass credentials down to the gh cli... it would be great if the access-tokens where available in the one or more of the phases... and as we use flakes arg and argstr are not an option.

In general I don't think exposing credentials to builders is the best approach because it's easy for builders to steal your credentials, or for users to steal credentials from (remote) builders. And you can already expose credentials to builders using extra-sandbox-paths or impureEnvVars, though this needs to be done on the daemon side.

I think what we'd ideally want is for builtin fetchers to support authentication. If the host was configured to require authentication, then fetching would be performed on the client side and the output passed to the daemon. If the daemon was remote, then this would involve a copy. This is exactly how it works with git+ssh:// and ssh agent today afaik.

The work @yes91 has done would be a big improvement already. Though ideally we'd want to support arbitrary headers (e.g. GitLab package registry expects a PRIVATE-TOKEN header). It would also be nice to be able to run external programs like pass or helpers that can fetch short-lived tokens on demand.

For example something similar to this:

In nix.conf:

access-tokens = my-gitlab.com:provider:my-gitlab-provider

In /usr/local/bin/my-gitlab-provider:

#!/usr/bin/env bash
token=$(pass show company/my-gitlab.com)
echo "PRIVATE-TOKEN: $token"

Nix would also want to allow interactivity if this required a password prompt, 2fa or something like that.

[ck3mp3r]

In general I don't think exposing credentials to builders is the best approach because it's easy for builders to steal your credentials, or for users to steal credentials from (remote) builders. And you can already expose credentials to builders using extra-sandbox-paths or impureEnvVars, though this needs to be done on the daemon side.

Our use case is literally nix develope github:corporation/flake-repo --access-tokens github.com=$(gh auth token) so that some users have easy access to some tools they need periodically without having to configure several different locations to support this. Because we want to save them from also having to compile the binaries we supply them via github releases, but it currently doesn't seem to be supported in this way... maybe we need some private binary caching like a corporate cachix... but that would require a load of other infrastructure. It would be ideal if we could just use github release assets for this. For my own project it works fine as the assets are not protected: github.com/ck3mp3r/rmx-cli...

[szlend]

My understanding is if you qualify your flake with github: then --access-tokens for GitHub should work. It just doesn't work with builtin fetchers like fetchurl.

[ck3mp3r]

My understanding is if you qualify your flake with github: then --access-tokens for GitHub should work. It just doesn't work with builtin fetchers like fetchurl.

That is correct, however, for repo access only, not for downloading release assets as they can only be fetched using fetchurl.

[fd]

Our use case is literally nix develope github:corporation/flake-repo --access-tokens github.com=$(gh auth token) so that some users have easy access to some tools they need periodically without having to configure several different locations to support this.

Something like access token helpers could help here. We currently use a system service to update the short-lived tokens while the developer has an active login session. Deeper credentials integration into nix seems like a good way to leak credentials into the nix store which is obviously a bad idea. Built-in fetchers (builtins.fetch*) could be able to use these helper, though all configuration should come from the nix-daemon (ie. builtins.fetch* should never be able to define a credentials helper., otherwise it would be trivial for a 3rd party flake to discover your credentials).

Authentication with binary caches could also use helpers in addition to the netrc file it currently supports.

There are many considerations here:

• is it okay to delegate credentials when doing a remote build? (limiting to builtins.fetch* might give direction here)
• are credentials visible to derivations (I sure hope not)
• where/how are credentials stored on the local system?
• should credentials impact the store path of a builtins.fetch* (it really shouldn't, so fixed-output derivations, aka pre-shared hashes are still required)

---

In some scenarios it's also easier to just, fetch, store and cache nix paths out-of-band. ie:

1. fetch with credentials, without nix
2. nix store add-path
3. cache the resulting nix-store path in a private binary cache.

[ck3mp3r]

I would expect --access-tokens github.com=$(gh auth token) to be enough to allow a fetcher to authenticate and download a private github release asset... the problem as stated above is that fetchurl gets stuck on the SSO page and happily thinks that is what it needs to download, in other words, --access-tokens doesn't seem to be sufficient. In our case the "helper" is the github cli i.e. gh

To get more adoption of Nix we need to keep the barrier for entry as low as possible, which means just installing nix and using a few nix develop github:coorp/flake-repo-with-some-useful-tool --access-tokens github.com=$(gh auth token) one-liners. We are not using Nix to build software locally, that is done in github using actions. Those actions also calculate the checksums, store them in a attribute set along with the download url and create a commit and merge in the corresponding repo, so when the flake is run, all the info is already present, only behind a private repo/release...

I guess this file best describes what I am trying to achieve, but in a private setting.

[fd]

To get more adoption of Nix we need to keep the barrier for entry as low as possible

I agree. Making Nix easier to use is definitely a priority. I think this is best achieved with educational content, for now. When browsing these discussions I get a big git ano 2009 vibe. Many people wanted to get in, but also struggled to map their expectations onto git at that time. Hah, remember CVS and SVN, quite a struggle that was...

To be absolutely clear this is not a you prioblem @ck3mp3r. It's very much an us issue. Just like git, nix is, probably, way to academic and abstract at it's core. I'm sure you know that. And I'm sure that some years down the road these things will be mostly non-issues. Nix is a fundamentals first kinda tool, this makes it harder to use, for now. Flakes help, but don't get us all the way.

If you want to chat about how we, in our org, solve these issue for now, feel free to get in touch.

[szlend]

Built-in fetchers (builtins.fetch) could be able to use these helper, though all configuration should come from the nix-daemon (ie. builtins.fetch should never be able to define a credentials helper., otherwise it would be trivial for a 3rd party flake to discover your credentials).

Yes, though any helper process should be spawned by the client talking to the daemon, otherwise it wont have access to the user's session (pass, system keychain, etc). This is already an issue with S3 substituters where you have to copy the AWS credentials to the root user which nix-daemon runs under. See https://github.com/NixOS/nix/issues/5723. That's why my suggestion was that the client should be able to perform fetching without involving the daemon and then add the path to the store. Very similar to your out-of-band workaround.

[ck3mp3r]

I have now tried --netrc-file, impureEnvVars, netrcPhase et al and am totally at a loss. Haven't found a single example that actually is reproducible and that works. Very disappointing to say the least.

[szlend]

They're all pretty bad yeah. Fyi netrc-file only works with plain http auth, so I don't think it's useful for GitHub. With impureEnvVars you need to start the nix-daemon with those variables which is not very convenient if you ever want to update them. It's useful in CI though. The most practical method on local machines is to create a credential file in e.g. /etc/nix/my-creds and add it to extra-sandbox-paths in /etc/nix/nix.conf. It might need a daemon restart the first time you set this option. Also make sure the file is readable to the nixbld group. Then you should be able to read the file in your derivation and build additional curlOpts for pkgs.fetchurl.

[ck3mp3r]

@szlend that is useful info, thank you, might be worth writing a devshell flake with the right cli-tools that helps with the setup. The target audience doesn't necessarily want to learn new tools, so it has to be a low barrier to entry...

[ck3mp3r]

This is what I've come up with in case someone needs a way to access private github releases... I'd love a more elegant builtin solution, but knowing Nix's limitations I think this is as good as it gets:

fetchrelease = { owner, repo, asset, version, sha256 }:
    pkgs.stdenv.mkDerivation {
      name = "${repo}-${asset}-${version}";
      phases = [ "buildPhase" ];
      nativeBuildInputs = with pkgs; [ gh ];
      outputHashMode = "recursive";
      outputHashAlgo = "sha256";
      outputHash = sha256;
      preferLocalBuild = true;
      buildPhase = ''
        export GH_TOKEN=$(grep -A 2 "machine github.com" "/etc/nix/netrc" | grep "password" | awk '{print $2}')
        gh release -R ${owner}/${repo} download ${version} -p ${asset}
        mkdir -p $out
        mv ${asset} $out/
      '';
    };

I've chosen to stick to the netrc format because it also doubles as the netrc file for normal operations. Let me know what you think and share improvements on the idea. I hope it helps someone from spending days and days trying to figure out a solution.

[Ericson2314]

A few related things

• https://github.com/NixOS/nix/issues/9077
• https://github.com/NixOS/nix/issues/9344
• https://github.com/NixOS/nix/pull/9429
• https://github.com/nixos/nix/tree/build-with-local-fs-store

CC @kolloch

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/will-nix-conf-options-set-in-commandline-be-forwarded-to-remote-builders/49968/2

[Atry]

This is what I've come up with in case someone needs a way to access private github releases... I'd love a more elegant builtin solution, but knowing Nix's limitations I think this is as good as it gets:

fetchrelease = { owner, repo, asset, version, sha256 }:
    pkgs.stdenv.mkDerivation {
      name = "${repo}-${asset}-${version}";
      phases = [ "buildPhase" ];
      nativeBuildInputs = with pkgs; [ gh ];
      outputHashMode = "recursive";
      outputHashAlgo = "sha256";
      outputHash = sha256;
      preferLocalBuild = true;
      buildPhase = ''
        export GH_TOKEN=$(grep -A 2 "machine github.com" "/etc/nix/netrc" | grep "password" | awk '{print $2}')
        gh release -R ${owner}/${repo} download ${version} -p ${asset}
        mkdir -p $out
        mv ${asset} $out/
      '';
    };

I've chosen to stick to the netrc format because it also doubles as the netrc file for normal operations. Let me know what you think and share improvements on the idea. I hope it helps someone from spending days and days trying to figure out a solution.

Would the nix builder in sandbox be able to access /etc/nix/netrc?

[Atry]

They're all pretty bad yeah. Fyi netrc-file only works with plain http auth, so I don't think it's useful for GitHub. With impureEnvVars you need to start the nix-daemon with those variables which is not very convenient if you ever want to update them. It's useful in CI though. The most practical method on local machines is to create a credential file in e.g. /etc/nix/my-creds and add it to extra-sandbox-paths in /etc/nix/nix.conf. It might need a daemon restart the first time you set this option. Also make sure the file is readable to the nixbld group. Then you should be able to read the file in your derivation and build additional curlOpts for pkgs.fetchurl.

I think it's not safe to set extra-sandbox-paths in /etc/nix/nix.conf because it would expose the secret to all nix build runs.

A more sophisticated approach is to save the secret to a temporary file and mount the temporary file via extra-sandbox-paths on fly.

Suppose you have a flake to download a file from a secret Gist URL:

{
  description = "A flake to download a file from a URL specified in /run/secret/url.txt";

  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-parts.url = "github:hercules-ci/flake-parts";
  };

  outputs = inputs@{ self, nixpkgs, flake-parts, ... }:
    flake-parts.lib.mkFlake { inherit inputs; } {
      systems = [ "x86_64-linux" ];

      perSystem = { system, pkgs, lib, ... }: {
        packages.default = pkgs.runCommand "download-file" {
          buildInputs = [ pkgs.curl pkgs.cacert ];
          outputHash = "sha256-6pZt7mxo+aLMFTfooiNnKkbp6oUlFEixZUQM1G/TLYA=";
        } ''
          url=$(cat /run/secret/url.txt)
          curl -o $out $url
        '';

      };
    };

}

Then the secret Gist URL can be specified in a temporary file mounted to /run/secret/url.txt.

# A secret Gist URL
SECRET='https://gist.githubusercontent.com/Atry/9cdedb8ad653f0441f4053281bdf3d2e/raw/79bcdd7213c31076288aa795f5d4b205d1a6f4d1/flake.lock'

SECRET_FILE_NAME="$(hexdump -n 16 -e '"%02x"' /dev/urandom)".txt

SECRET_DIR="$(mktemp -d -p /tmp)"
# Only a process that knows SECRET_FILE_NAME can access the file.
chmod o+x "$SECRET_DIR"

printf %s "$SECRET" > "$SECRET_DIR"/"$SECRET_FILE_NAME"

nix build --option extra-sandbox-paths /run/secret/url.txt="$SECRET_DIR"/"$SECRET_FILE_NAME"

[httpdev]

This is what I've come up with in case someone needs a way to access private github releases... I'd love a more elegant builtin solution, but knowing Nix's limitations I think this is as good as it gets:

fetchrelease = { owner, repo, asset, version, sha256 }:
    pkgs.stdenv.mkDerivation {
      name = "${repo}-${asset}-${version}";
      phases = [ "buildPhase" ];
      nativeBuildInputs = with pkgs; [ gh ];
      outputHashMode = "recursive";
      outputHashAlgo = "sha256";
      outputHash = sha256;
      preferLocalBuild = true;
      buildPhase = ''
        export GH_TOKEN=$(grep -A 2 "machine github.com" "/etc/nix/netrc" | grep "password" | awk '{print $2}')
        gh release -R ${owner}/${repo} download ${version} -p ${asset}
        mkdir -p $out
        mv ${asset} $out/
      '';
    };

I've chosen to stick to the netrc format because it also doubles as the netrc file for normal operations. Let me know what you think and share improvements on the idea. I hope it helps someone from spending days and days trying to figure out a solution.

I've been banging my head a while at trying to get fetchurlBoot to pick up my netrc-file set using netrc-file in my user's nix.conf as described here. I finally found that it does work if I build to a local store (--store /home/user/store), but not if the build is passed on to the daemon. Did anyone have a similar problem with the solutions posted above? Creating a global .netrc with all users' credentials would be "slightly" on the insecure side.

[szlend]

Since Nix 2.19 you can also use the experimental feature configurable-impure-env to pass impure env vars from the client to the daemon.

For example:

NIX_CONIFG="impure-env = TOKEN=$(fetch-token-from-key-store)" nix build .#foo

This requires your user to be in trusted-users and the FOD builder to set impureEnvVars.

Maybe we could allow untrusted users as well? Discussion in #10472.