Allow all locked flake inputs in restricted eval mode

[edolstra]

Currently, with --restrict-eval, building a flake fails if it depends on a non-whitelisted input, e.g.

in job ‘x86_64-darwin.hello’:
error: access to URI 'https://git.savannah.gnu.org/git/hello.git' is forbidden in restricted mode

Maybe this restriction shouldn't apply to locked inputs, since we know their hash? Alternatively, maybe Hydra shouldn't use --restrict-eval for locked flakes.

[sbourdeauducq]

How can I whitelist an input as a workaround?

[sbourdeauducq]

Found this hacky solution: https://git.m-labs.hk/M-Labs/it-infra/commit/82e161dba3240c7ef2676ab6bea9ab78cab2068b - not sure if there's a better way to do it.

[edolstra]

@sbourdeauducq On hydra.nixos.org we have:

  nix.extraOptions =
    ''
      allowed-uris = https://github.com/ https://git.savannah.gnu.org/
    '';

[chkno]

Simple, concrete copy/pastable reproduction recipe (no hydra needed):

nix-shell -p nixUnstable
mkdir 5039
cd 5039
cat  >flake.nix <<EOF
{
  inputs.nixpkgs = {

    # This one works
    #url = "github:NixOS/nixpkgs";

    # These all fail with "error: access to ... is forbidden in restricted mode"
    #url = "git+https:///github.com/NixOS/nixpkgs?ref=nixos-unstable&rev=2fa862644fc15ecb525eb8cd0a60276f1c340c7c";
    url = "git+https:///github.com/NixOS/nixpkgs?ref=nixos-unstable";
    #url = "git+https:///github.com/NixOS/nixpkgs";
    #url = "git+file:///home/user/devel/nixpkgs";
    #url = "/home/user/devel/nixpkgs";
  };

  outputs = { self, nixpkgs }: {
    nixosConfigurations = {
      foo = nixpkgs.legacyPackages.x86_64-linux.nixos ({ modulesPath, ...}: {
        imports = [ (modulesPath + "/virtualisation/qemu-vm.nix") ];
      });
    };
  };
}
EOF
nix --option restrict-eval true --experimental-features 'nix-command flakes' flake check

Results in:

error: access to URI 'https:///github.com/nixos/nixpkgs' is forbidden in restricted mode

I'm confused why fetchTree as invoked by call-flake.nix is accessing the input URL after the flake-input locking has completed and everything has already been copied into the store. Example trace:

$ gdb nix
...
(gdb) break checkSourcePath
(gdb) run --option restrict-eval true --experimental-features 'nix-command flakes' flake check -vvvvvvvvvvvv
...
copying '/home/user/5039'...
acquiring global GC lock '/nix/var/nix/gc.lock'
acquiring read lock on '/nix/var/nix/temproots/2596104'
acquiring write lock on '/nix/var/nix/temproots/2596104'
downgrading to read lock on '/nix/var/nix/temproots/2596104'
got tree '/nix/store/pj1ar5cpsl0m1qb1hcisf9isrf9wkflm-source' from 'path:/home/user/5039?narHash=sha256-kOLsokbrXgI1qQBR%2fP3xIaM39g4aA0R4jiMCDJno4j8='

<continue from the break point a few times>

checking access to '/nix/store/pj1ar5cpsl0m1qb1hcisf9isrf9wkflm-source/flake.nix'
evaluating file '/nix/store/pj1ar5cpsl0m1qb1hcisf9isrf9wkflm-source/flake.nix'
old lock file: {
  "nodes": {
    "nixpkgs": {
      "locked": {
        "lastModified": 1634782485,
        "narHash": "sha256-psfh4OQSokGXG0lpq3zKFbhOo3QfoeudRcaUnwMRkQo=",
        "ref": "qemu-vm-isolation",
        "rev": "34ad3ffe08adfca17fcb4e4a47bb5f3b113687be",
        "revCount": 324423,
        "type": "git",
        "url": "file:///home/user/devel/nixpkgs"
      },
      "original": {
        "type": "git",
        "url": "file:///home/user/devel/nixpkgs"
      }
    },
    "root": {
      "inputs": {
        "nixpkgs": "nixpkgs"
      }
    }
  },
  "root": "root",
  "version": 7
}
computing lock file node ''
computing input 'nixpkgs'
keeping existing input 'nixpkgs'
computing lock file node 'nixpkgs'
new lock file: {
  "nodes": {
    "nixpkgs": {
      "locked": {
        "lastModified": 1634782485,
        "narHash": "sha256-psfh4OQSokGXG0lpq3zKFbhOo3QfoeudRcaUnwMRkQo=",
        "ref": "qemu-vm-isolation",
        "rev": "34ad3ffe08adfca17fcb4e4a47bb5f3b113687be",
        "revCount": 324423,
        "type": "git",
        "url": "file:///home/user/devel/nixpkgs"
      },
      "original": {
        "type": "git",
        "url": "file:///home/user/devel/nixpkgs"
      }
    },
    "root": {
      "inputs": {
        "nixpkgs": "nixpkgs"
      }
    }
  },
  "root": "root",
  "version": 7
}
evaluating flake...
evaluating flake
checking flake output 'nixosConfigurations'...
checking NixOS configuration 'nixosConfigurations.foo'...
checking NixOS configuration 'nixosConfigurations.foo'

Thread 1 "nix" hit Breakpoint 1, 0x00007ffff7e9df70 in nix::EvalState::checkSourcePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /nix/store/jnx8zlprxamvqnx2lg7vbkpl51gr56pb-nix-2.4pre20211006_53e4794/lib/libnixexpr.so
(gdb) bt
#0  0x00007ffff7e9df70 in nix::EvalState::checkSourcePath(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from libnixexpr.so
#1  0x00007ffff7e9ea10 in nix::EvalState::checkURI(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from libnixexpr.so
#2  0x00007ffff7f5173d in nix::fixURI(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nix::EvalState&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from libnixexpr.so
#3  0x00007ffff7f542b8 in nix::fixURIForGit(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, nix::EvalState&) () from libnixexpr.so
#4  0x00007ffff7f558ae in nix::fetchTree(nix::EvalState&, nix::Pos const&, nix::Value**, nix::Value&, std::optional<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, nix::FetchTreeParams const&) () from libnixexpr.so
#5  0x00007ffff7f55b05 in nix::prim_fetchTree(nix::EvalState&, nix::Pos const&, nix::Value**, nix::Value&) () from libnixexpr.so
#6  0x00007ffff7e9af35 in nix::EvalState::callPrimOp(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#7  0x00007ffff7ea0c33 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#8  0x00007ffff7ea0ddd in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#9  0x00007ffff7ea1744 in nix::ExprVar::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#10 0x00007ffff7ea4ab7 in nix::ExprConcatStrings::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#11 0x00007ffff7ea4ab7 in nix::ExprConcatStrings::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#12 0x00007ffff7ea4ab7 in nix::ExprConcatStrings::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#13 0x0000000000581148 in nix::EvalState::forceValue(nix::Value&, nix::Pos const&) ()
#14 0x00007ffff7ea2ac0 in nix::EvalState::coerceToString(nix::Pos const&, nix::Value&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&, bool, bool, bool) () from libnixexpr.so
#15 0x00007ffff7ea33c9 in nix::EvalState::coerceToPath(nix::Pos const&, nix::Value&, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from libnixexpr.so
#16 0x00007ffff7f3f7b3 in nix::import(nix::EvalState&, nix::Pos const&, nix::Value&, nix::Value*, nix::Value&) () from libnixexpr.so
#17 0x00007ffff7e9af35 in nix::EvalState::callPrimOp(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#18 0x00007ffff7ea0c33 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#19 0x00007ffff7ea0ddd in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#20 0x00007ffff7ea1744 in nix::ExprVar::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#21 0x00007ffff7ea345e in nix::ExprSelect::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#22 0x00007ffff7f305dd in nix::prim_isFunction(nix::EvalState&, nix::Pos const&, nix::Value**, nix::Value&) () from libnixexpr.so
#23 0x00007ffff7e9af35 in nix::EvalState::callPrimOp(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#24 0x00007ffff7ea0c33 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#25 0x00007ffff7ea0ddd in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#26 0x00007ffff7ea3db6 in nix::ExprAssert::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#27 0x00007ffff7ea0c52 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#28 0x00007ffff7ea36d4 in nix::ExprSelect::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#29 0x00007ffff7ea0c52 in nix::EvalState::callFunction(nix::Value&, nix::Value&, nix::Value&, nix::Pos const&) () from libnixexpr.so
#30 0x00007ffff7ea17a5 in nix::ExprVar::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#31 0x00007ffff7ea345e in nix::ExprSelect::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#32 0x00007ffff7ea0db8 in nix::ExprApp::eval(nix::EvalState&, nix::Env&, nix::Value&) () from libnixexpr.so
#33 0x00007ffff7ea1258 in nix::EvalState::autoCallFunction(nix::Bindings&, nix::Value&, nix::Value&) () from libnixexpr.so
#34 0x00007ffff7e700bf in nix::findAlongAttrPath(nix::EvalState&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Bindings&, nix::Value&) () from libnixexpr.so
#35 0x0000000000587d69 in CmdFlakeCheck::run(nix::ref<nix::Store>)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value&, nix::Pos const&)#8}::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value&, nix::Pos const&) const ()
#36 0x0000000000589ed5 in CmdFlakeCheck::run(nix::ref<nix::Store>)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value&, nix::Pos const&)#11}::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, nix::Value&, nix::Pos const&) const ()
#37 0x0000000000585f8b in CmdFlakeCheck::run(nix::ref<nix::Store>) ()
#38 0x00007ffff74e9050 in nix::StoreCommand::run() () from libnixcmd.so
#39 0x000000000059f81f in nix::mainWrapped(int, char**) ()
#40 0x00007ffff7b3f678 in nix::handleExceptions(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::function<void ()>) () from libnixmain.so
#41 0x000000000047a4c6 in main ()
(gdb) c
Continuing.
error: access to path '/home/user/devel/nixpkgs' is forbidden in restricted mode
(use '--show-trace' to show detailed location information)
[Inferior 1 (process 2596069) exited with code 01]

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/flake-restricted-evaluation-how-to-download-stuff/17105/5

[t184256]

I've found out that another workaround besides allowed-uris, one that doesn't require modifying nix configuration of your users would be to use derivation = { builder = "builtin:fetchurl"; ...}.

[chkno]

Expanding for clarity: In @t184256's example linked from the the discourse thread, the flake defines no flake inputs at all, and then gets the actual inputs via allowed-in-pure-mode derivation { builder = "builtin:fetchurl"; ... } derivations. I.e., this workaround avoids using flake-inputs, rather than making flake-inputs work somehow.

[typetetris]

not stale, still an issue for us at least it is very unexpected and unintuitive

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/hydra-hash-and-inputs/21133/8

[jrobsonchase]

I was briefly bitten by this after updating to a version of nixos-unstable after https://github.com/NixOS/nixpkgs/pull/282441 landed.

Had to add nix.settings.allowed-uris = "github: gitlab: git+ssh://"; to get things working again. It was unnecessary before, since the patch had disabled restricted mode altogether.

[jrobsonchase]

Just found another related sharp edge. Details of transitive dependencies are "leaked", i.e. setting allowed-uris to include direct dependencies for your flakes is insufficient - you need to recursively add them for the entire dependency tree.

It also feels wrong that working around this for a single flake by adding things to allowed-uris applies to all flakes.

I found that you can also set accept-flake-config = true in your global nix.conf, and then add nixConfig.allowed-uris in individual flakes, but that just feels like "allowed locked inputs" with extra steps.