hercules-ci / hercules-ci-agent

https://hercules-ci.com build and deployment agent
Apache License 2.0
99 stars 19 forks source link

Ability to pass NIX_PATH to executors #188

Closed utdemir closed 2 years ago

utdemir commented 4 years ago

Hi,

I am trying to to build my dotfiles repository with Hercules CI.

I have nixpkgs pinned, but still some of my dependencies, use NIX_PATH to lookup some trivial files from nixpkgs. So, I currently get these kind of errors error when used with Hercules CI:

Exception: file 'nixpkgs/nixos/modules/misc/assertions.nix' was not found in the Nix search path (add it using $NIX_PATH or -I), at /nix/store/k1pc4wx9f69l56kl8g0sxkzpwviw4x0i-source/modules/modules.nix:159:17; type: nix::ThrownError

error: file 'nixpkgs/nixos/modules/installer/scan/not-detected.nix' was not found in the Nix search path (add it using $NIX_PATH or -I), at /etc/nixos/hardware-configuration.nix:8:7

I was able to get things working by patching things and adding conditionals for CI, but it isn't pretty. Ideally, I would prefer having NIX_PATH available on executors, since there's still tooling using it. I know that it makes the builds less deterministic, so maybe having an option with a warning might be a better option.

Another alternative would be to have a way to set NIX_PATH on ci.nix. Maybe with some special key like:

# ci.nix:
{
  ...
  "hercules-ci-meta": {
    NIX_PATH = "nixpkgs=${(import ./sources.nix).nixpkgs}";
  }
}
domenkozar commented 4 years ago

Yeah, we discussed that supporting NIX_PATH as an opt-in makes sense.

utdemir commented 4 years ago

Thank you @domenkozar .

I just wanted to note that what I was asking for will (probably) also be possible with the recent "recursive nix" feature. After a new Nix version is released and if the agent supports it, I should be able to do something like:

runCommand "build" {} ''
  export NIX_PATH=nixpkgs="${sources.nix}"
  nix-build ./default.nix
''
roberth commented 2 years ago

We weren't able to prioritize this at the time and it seems that the window for this to be useful is closing, as the ecosystem has been moving away from NIX_PATH, accelerated by the adoption of tools like niv and the advent of Flakes. With version 0.9, the agent is in a good position to support this, as it splits the evaluation of project configuration and the actual evaluation for building. Do you still need this?

utdemir commented 2 years ago

Hi @roberth. I haven't been using Hercules CI for a while, so I do not need it.

And I also agree that NIX_PATH is something we should move away from, so I'm happy to close it. Thanks!