Open simonzkl opened 1 year ago
This is a major blocker for adopting nix at my organization. I've made a recent push to use nix for our internal projects, but the only thing standing in the way is that we are hosting some binary inputs on a private Artifactory instance. There are multiple authentication methods but they all require setting http headers for fetchurl. I have sketched a solution that extends the tarball fetcher with support for access-tokens, but this feels like a hack and is far from ideal for the reasons described above. Since flakes depend on libfetchers, I feel this is also an extremely important feature for stabilizing flakes.
This is also blocking our internal adoption as we cannot write derivations that can download github release assets, fetchurl actually just downloads the SSO page and thinks that is the requested target.
I am not sure how far --access-tokens and --extra-access-tokens go down the chain... one idea I had was to implement a custom fetcher using the gh cli, but somehow I need to pass credentials down to the gh cli... it would be great if the access-tokens where available in the one or more of the phases... and as we use flakes arg and argstr are not an option.
I am not sure how far --access-tokens and --extra-access-tokens go down the chain... one idea I had was to implement a custom fetcher using the gh cli, but somehow I need to pass credentials down to the gh cli... it would be great if the access-tokens where available in the one or more of the phases... and as we use flakes arg and argstr are not an option.
In general I don't think exposing credentials to builders is the best approach because it's easy for builders to steal your credentials, or for users to steal credentials from (remote) builders. And you can already expose credentials to builders using extra-sandbox-paths
or impureEnvVars
, though this needs to be done on the daemon side.
I think what we'd ideally want is for builtin fetchers to support authentication. If the host was configured to require authentication, then fetching would be performed on the client side and the output passed to the daemon. If the daemon was remote, then this would involve a copy. This is exactly how it works with git+ssh://
and ssh agent today afaik.
The work @yes91 has done would be a big improvement already. Though ideally we'd want to support arbitrary headers (e.g. GitLab package registry expects a PRIVATE-TOKEN
header). It would also be nice to be able to run external programs like pass
or helpers that can fetch short-lived tokens on demand.
For example something similar to this:
In nix.conf
:
access-tokens = my-gitlab.com:provider:my-gitlab-provider
In /usr/local/bin/my-gitlab-provider
:
#!/usr/bin/env bash
token=$(pass show company/my-gitlab.com)
echo "PRIVATE-TOKEN: $token"
Nix would also want to allow interactivity if this required a password prompt, 2fa or something like that.
In general I don't think exposing credentials to builders is the best approach because it's easy for builders to steal your credentials, or for users to steal credentials from (remote) builders. And you can already expose credentials to builders using
extra-sandbox-paths
orimpureEnvVars
, though this needs to be done on the daemon side.
Our use case is literally nix develope github:corporation/flake-repo --access-tokens github.com=$(gh auth token)
so that some users have easy access to some tools they need periodically without having to configure several different locations to support this. Because we want to save them from also having to compile the binaries we supply them via github releases, but it currently doesn't seem to be supported in this way... maybe we need some private binary caching like a corporate cachix... but that would require a load of other infrastructure. It would be ideal if we could just use github release assets for this.
For my own project it works fine as the assets are not protected: github.com/ck3mp3r/rmx-cli...
My understanding is if you qualify your flake with github:
then --access-tokens
for GitHub should work. It just doesn't work with builtin fetchers like fetchurl
.
My understanding is if you qualify your flake with
github:
then--access-tokens
for GitHub should work. It just doesn't work with builtin fetchers likefetchurl
.
That is correct, however, for repo access only, not for downloading release assets as they can only be fetched using fetchurl.
Our use case is literally
nix develope github:corporation/flake-repo --access-tokens github.com=$(gh auth token)
so that some users have easy access to some tools they need periodically without having to configure several different locations to support this.
Something like access token helpers could help here. We currently use a system service to update the short-lived tokens while the developer has an active login session. Deeper credentials integration into nix seems like a good way to leak credentials into the nix store which is obviously a bad idea. Built-in fetchers (builtins.fetch*
) could be able to use these helper, though all configuration should come from the nix-daemon (ie. builtins.fetch*
should never be able to define a credentials helper., otherwise it would be trivial for a 3rd party flake to discover your credentials).
Authentication with binary caches could also use helpers in addition to the netrc file it currently supports.
There are many considerations here:
builtins.fetch*
might give direction here)builtins.fetch*
(it really shouldn't, so fixed-output derivations, aka pre-shared hashes are still required)In some scenarios it's also easier to just, fetch, store and cache nix paths out-of-band. ie:
I would expect --access-tokens github.com=$(gh auth token)
to be enough to allow a fetcher to authenticate and download a private github release asset... the problem as stated above is that fetchurl gets stuck on the SSO page and happily thinks that is what it needs to download, in other words, --access-tokens doesn't seem to be sufficient.
In our case the "helper" is the github cli i.e. gh
To get more adoption of Nix we need to keep the barrier for entry as low as possible, which means just installing nix and using a few nix develop github:coorp/flake-repo-with-some-useful-tool --access-tokens github.com=$(gh auth token)
one-liners. We are not using Nix to build software locally, that is done in github using actions. Those actions also calculate the checksums, store them in a attribute set along with the download url and create a commit and merge in the corresponding repo, so when the flake is run, all the info is already present, only behind a private repo/release...
I guess this file best describes what I am trying to achieve, but in a private setting.
To get more adoption of Nix we need to keep the barrier for entry as low as possible
I agree. Making Nix easier to use is definitely a priority. I think this is best achieved with educational content, for now. When browsing these discussions I get a big git ano 2009 vibe. Many people wanted to get in, but also struggled to map their expectations onto git at that time. Hah, remember CVS and SVN, quite a struggle that was...
To be absolutely clear this is not a you prioblem @ck3mp3r. It's very much an us issue. Just like git, nix is, probably, way to academic and abstract at it's core. I'm sure you know that. And I'm sure that some years down the road these things will be mostly non-issues. Nix is a fundamentals first kinda tool, this makes it harder to use, for now. Flakes help, but don't get us all the way.
If you want to chat about how we, in our org, solve these issue for now, feel free to get in touch.
Built-in fetchers (builtins.fetch) could be able to use these helper, though all configuration should come from the nix-daemon (ie. builtins.fetch should never be able to define a credentials helper., otherwise it would be trivial for a 3rd party flake to discover your credentials).
Yes, though any helper process should be spawned by the client talking to the daemon, otherwise it wont have access to the user's session (pass, system keychain, etc). This is already an issue with S3 substituters where you have to copy the AWS credentials to the root
user which nix-daemon runs under. See https://github.com/NixOS/nix/issues/5723. That's why my suggestion was that the client should be able to perform fetching without involving the daemon and then add the path to the store. Very similar to your out-of-band workaround.
I have now tried --netrc-file, impureEnvVars, netrcPhase et al and am totally at a loss. Haven't found a single example that actually is reproducible and that works. Very disappointing to say the least.
They're all pretty bad yeah. Fyi netrc-file
only works with plain http auth, so I don't think it's useful for GitHub. With impureEnvVars
you need to start the nix-daemon with those variables which is not very convenient if you ever want to update them. It's useful in CI though. The most practical method on local machines is to create a credential file in e.g. /etc/nix/my-creds
and add it to extra-sandbox-paths
in /etc/nix/nix.conf
. It might need a daemon restart the first time you set this option. Also make sure the file is readable to the nixbld
group. Then you should be able to read the file in your derivation and build additional curlOpts
for pkgs.fetchurl
.
@szlend that is useful info, thank you, might be worth writing a devshell flake with the right cli-tools that helps with the setup. The target audience doesn't necessarily want to learn new tools, so it has to be a low barrier to entry...
This is what I've come up with in case someone needs a way to access private github releases... I'd love a more elegant builtin solution, but knowing Nix's limitations I think this is as good as it gets:
fetchrelease = { owner, repo, asset, version, sha256 }:
pkgs.stdenv.mkDerivation {
name = "${repo}-${asset}-${version}";
phases = [ "buildPhase" ];
nativeBuildInputs = with pkgs; [ gh ];
outputHashMode = "recursive";
outputHashAlgo = "sha256";
outputHash = sha256;
preferLocalBuild = true;
buildPhase = ''
export GH_TOKEN=$(grep -A 2 "machine github.com" "/etc/nix/netrc" | grep "password" | awk '{print $2}')
gh release -R ${owner}/${repo} download ${version} -p ${asset}
mkdir -p $out
mv ${asset} $out/
'';
};
I've chosen to stick to the netrc format because it also doubles as the netrc file for normal operations. Let me know what you think and share improvements on the idea. I hope it helps someone from spending days and days trying to figure out a solution.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
This is what I've come up with in case someone needs a way to access private github releases... I'd love a more elegant builtin solution, but knowing Nix's limitations I think this is as good as it gets:
fetchrelease = { owner, repo, asset, version, sha256 }: pkgs.stdenv.mkDerivation { name = "${repo}-${asset}-${version}"; phases = [ "buildPhase" ]; nativeBuildInputs = with pkgs; [ gh ]; outputHashMode = "recursive"; outputHashAlgo = "sha256"; outputHash = sha256; preferLocalBuild = true; buildPhase = '' export GH_TOKEN=$(grep -A 2 "machine github.com" "/etc/nix/netrc" | grep "password" | awk '{print $2}') gh release -R ${owner}/${repo} download ${version} -p ${asset} mkdir -p $out mv ${asset} $out/ ''; };
I've chosen to stick to the netrc format because it also doubles as the netrc file for normal operations. Let me know what you think and share improvements on the idea. I hope it helps someone from spending days and days trying to figure out a solution.
Would the nix builder in sandbox be able to access /etc/nix/netrc
?
They're all pretty bad yeah. Fyi
netrc-file
only works with plain http auth, so I don't think it's useful for GitHub. WithimpureEnvVars
you need to start the nix-daemon with those variables which is not very convenient if you ever want to update them. It's useful in CI though. The most practical method on local machines is to create a credential file in e.g./etc/nix/my-creds
and add it toextra-sandbox-paths
in/etc/nix/nix.conf
. It might need a daemon restart the first time you set this option. Also make sure the file is readable to thenixbld
group. Then you should be able to read the file in your derivation and build additionalcurlOpts
forpkgs.fetchurl
.
I think it's not safe to set extra-sandbox-paths
in /etc/nix/nix.conf
because it would expose the secret to all nix build
runs.
A more sophisticated approach is to save the secret to a temporary file and mount the temporary file via extra-sandbox-paths
on fly.
Suppose you have a flake to download a file from a secret Gist URL:
{
description = "A flake to download a file from a URL specified in /run/secret/url.txt";
inputs = {
nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
flake-parts.url = "github:hercules-ci/flake-parts";
};
outputs = inputs@{ self, nixpkgs, flake-parts, ... }:
flake-parts.lib.mkFlake { inherit inputs; } {
systems = [ "x86_64-linux" ];
perSystem = { system, pkgs, lib, ... }: {
packages.default = pkgs.runCommand "download-file" {
buildInputs = [ pkgs.curl pkgs.cacert ];
outputHash = "sha256-6pZt7mxo+aLMFTfooiNnKkbp6oUlFEixZUQM1G/TLYA=";
} ''
url=$(cat /run/secret/url.txt)
curl -o $out $url
'';
};
};
}
Then the secret Gist URL can be specified in a temporary file mounted to /run/secret/url.txt
.
# A secret Gist URL
SECRET='https://gist.githubusercontent.com/Atry/9cdedb8ad653f0441f4053281bdf3d2e/raw/79bcdd7213c31076288aa795f5d4b205d1a6f4d1/flake.lock'
SECRET_FILE_NAME="$(hexdump -n 16 -e '"%02x"' /dev/urandom)".txt
SECRET_DIR="$(mktemp -d -p /tmp)"
# Only a process that knows SECRET_FILE_NAME can access the file.
chmod o+x "$SECRET_DIR"
printf %s "$SECRET" > "$SECRET_DIR"/"$SECRET_FILE_NAME"
nix build --option extra-sandbox-paths /run/secret/url.txt="$SECRET_DIR"/"$SECRET_FILE_NAME"
Is your feature request related to a problem? Please describe.
We desperately need some sort of credentials provider support for
builtins.fetch*
functions. The best you can currently do is to usepkgs.fetchurl
, expose your credentials in plain text at some globally accessible path like/etc/nix/my-creds
and add it toextra-sandbox-paths
. You can restrict access to onlynixbld
though it's not like it matters because anyone who has access to the nix builder can echo the credentials and fetch them from the build log.Describe the solution you'd like
pass
or the system keychain).Describe alternatives you've considered
access-tokens
currently doesn't work, is limited to credentials being exposed in plain text, assumes the credentials don't need to be refreshed and is limited to oauth/pat for specific platforms like gitlab/github.Somewhat related
Priorities
Add :+1: to issues you find important.