search

NixOS / hydra

Hydra, the Nix-based continuous build system
http://nixos.org/hydra
GNU General Public License v3.0
1.19k stars 302 forks source link

nixpkgs.nixos throws hydra-eval-jobs into an infinite loop #763

Closed dudebout closed 4 years ago

dudebout commented 4 years ago

Issue Evaluating NixOS system configurations built with nixpkgs.nixos ARGS works via nix-instantiate but throws hydra-eval-jobs in an infinite recursion loop. hydra-eval-jobs generates a growing list of packages of the following form:

MY_NAME._module.args.pkgs.PKG1
MY_NAME._module.args.pkgs.__splicedPackages.PKG1
MY_NAME._module.args.pkgs.__splicedPackages.__splicedPackages.PKG1
MY_NAME._module.args.pkgs.__splicedPackages.[...].__splicedPackages.PKG1

nixpkgs or hydra issue I am not sure if this is a hydra issue, or an issue with nixpkgs.nixos. Opening an issue against hydra as a first step, but I can open one against nixpkgs if this is more appropriate.

Background I was trying to use nixpkgs.nixos based on the Mayflower blog post Building Customised NixOS Images. By switching to:

(import (<nixpkgs/nixos>) { configuration = /my-configuration.nix; }).system

I was able to build my configuration in and out of hydra.

Troubleshooting I tried and figure out what was going on, but was not able to get to the bottom of it. During my investigation, I found the following comment in splice.nix:

# For performance reasons, rather than uniformally splice in all cases, we only
# do so when `pkgs` and `buildPackages` are distinct. The `actuallySplice`
# parameter there the boolean value of that equality check.

and I am wondering if pkgs and buildPackages are always different when using hydra-eval-jobs.

Minimal reproducer Here is a minimal nix expression showing the behavior:

{ nixpkgs }:
let
  pkgs = import nixpkgs {};
in {
  minimalNixos = pkgs.nixos [
    { fileSystems."/" =
        { device = "/dev/sda";
          fsType = "ext4";
        };
      boot.loader.grub = {
        enable = true;
        version = 2;
        device = "/dev/sda";
      };
    }
  ];
}

I saved this file under /tmp/minimal-nixos.nix and ran the following two commands:

$ nix-build \
--arg nixpkgs /nix/store/q6ppzkyplaz92x2p3pyibid9qybml1wr-source \
/tmp/minimal-nixos.nix \
-A minimalNixos
$ sudo -u hydra /nix/store/adwmf891m560ppx8f4170a56yp89vi6z-hydra-2019-08-30/bin/hydra-eval-jobs \
--gc-roots-dir /nix/var/nix/gcroots/hydra \
-j 1 \        
-I path1=/nix/store/q6ppzkyplaz92x2p3pyibid9qybml1wr-source \
-I path2=/tmp \
--arg nixpkgs /nix/store/q6ppzkyplaz92x2p3pyibid9qybml1wr-source \
/tmp/minimal-nixos.nix

(As far as I understand, the -I path1 and -I path2 options are required because hydra runs in restricted mode and does not want to consume paths which are not accessible via NIX_PATH) The second command generates output of the form:

{
  "minimalNixos._module.args.baseModules": {
    "error": "unsupported value: [ /nix/store/q6ppzkyplaz92x2p3pyibid9qybml1wr-source/nixos/modules/config/debug-info.nix /nix/store/q6ppzkyplaz92x2p3pyibid9qybml1wr-source/nixos/modules/config/fonts/fontconfig.nix ... ]
  },
  "minimalNixos._module.args.extraModules": {
    "error": "unsupported value: [ ]"
  },
  "minimalNixos._module.args.modules": {
    "error": "unsupported value: [ <LAMBDA> { boot = { loader = { grub = { device = \"/dev/sda\"; enable = true; version = 2; }; }; }; fileSystems = <CODE>; } ]"
  },
  "minimalNixos._module.args.pkgs.AAAAAASomeThingsFailToEvaluate": {
    "error": "Please be informed that this pseudo-package is not the only part of\nNixpkgs that fails to evaluate. You should not evaluate entire Nixpkgs\nwithout some special measures to handle failing packages, like those taken\nby Hydra.\n"
  },
  "minimalNixos._module.args.pkgs.AMB-plugins": {
    "nixName": "AMB-plugins-0.8.1",
    "system": "x86_64-linux",
    "drvPath": "/nix/store/8nl6236bib486pmhdcph6bgdwrb36g5z-AMB-plugins-0.8.1.drv",
    "description": "A set of ambisonics ladspa plugins",
    "license": "gpl2Plus",
    "homepage": "http://kokkinizita.linuxaudio.org/linuxaudio/ladspa/index.html",
    "maintainers": "bart@magnetophon.nl",
    "schedulingPriority": 100,
    "timeout": 36000,
    "maxSilent": 7200,
    "isChannel": false,
    "outputs": {
      "out": "/nix/store/6a278kldhpj8f1lbdzkcmks0595cigmi-AMB-plugins-0.8.1"
    }
  },
[...]
  "minimalNixos._module.args.pkgs._9pfs": {
    "nixName": "9pfs-20150918",
    "system": "x86_64-linux",
    "drvPath": "/nix/store/xmjagsxiz7hjrbpkjz83wfl1yjzpdpni-9pfs-20150918.drv",
    "description": "FUSE-based client of the 9P network filesystem protocol",
    "license": "lpl-102, bsd2",
    "homepage": "https://github.com/mischief/9pfs",
    "maintainers": "edolstra+nixpkgs@gmail.com",
    "schedulingPriority": 100,
    "timeout": 36000,
    "maxSilent": 7200,
    "isChannel": false,
    "outputs": {
      "out": "/nix/store/g1hdnllrldiryidp743l56s3naqkmmq4-9pfs-20150918"
    }
  },
  "minimalNixos._module.args.pkgs.__splicedPackages.AAAAAASomeThingsFailToEvaluate": {
    "error": "Please be informed that this pseudo-package is not the only part of\nNixpkgs that fails to evaluate. You should not evaluate entire Nixpkgs\nwithout some special measures to handle failing packages, like those taken\nby Hydra.\n"
  },
  "minimalNixos._module.args.pkgs.__splicedPackages.AMB-plugins": {
    "nixName": "AMB-plugins-0.8.1",
    "system": "x86_64-linux",
    "drvPath": "/nix/store/8nl6236bib486pmhdcph6bgdwrb36g5z-AMB-plugins-0.8.1.drv",
    "description": "A set of ambisonics ladspa plugins",
    "license": "gpl2Plus",
    "homepage": "http://kokkinizita.linuxaudio.org/linuxaudio/ladspa/index.html",
    "maintainers": "bart@magnetophon.nl",
    "schedulingPriority": 100,
    "timeout": 36000,
    "maxSilent": 7200,
    "isChannel": false,
    "outputs": {
      "out": "/nix/store/6a278kldhpj8f1lbdzkcmks0595cigmi-AMB-plugins-0.8.1"
    }
  },
[...]

The lines showing the beginning of the infinite loop are:

  "minimalNixos._module.args.pkgs.AAAAAASomeThingsFailToEvaluate": {
  "minimalNixos._module.args.pkgs.AMB-plugins": {
[...]
  "minimalNixos._module.args.pkgs._9pfs": {
  "minimalNixos._module.args.pkgs.__splicedPackages.AAAAAASomeThingsFailToEvaluate": {
  "minimalNixos._module.args.pkgs.__splicedPackages.AMB-plugins": {
[...]
  "minimalNixos._module.args.pkgs.__splicedPackages._9pfs": {
  "minimalNixos._module.args.pkgs.__splicedPackages.__splicedPackages.AAAAAASomeThingsFailToEvaluate": {
  "minimalNixos._module.args.pkgs.__splicedPackages.__splicedPackages.AMB-plugins": {
edolstra commented 4 years ago

This is expected behavior since nixos doesn't return a derivation (i.e. Hydra job) but a configuration. You'd have to select some attribute that you want to build (like system.build.toplevel).

dudebout commented 4 years ago

Thank you Eelco for looking into this. Could you point me at some documentation to grasp why nix-instantiate and nix-build are able to evaluate and build that expression but not hydra? Do these tools have a default behavior that hydra does not have for configurations?

edolstra commented 4 years ago

I think nix-build doesn't recurse, so it only builds the top-level derivation attributes. nixos is defined as c.config.system.build // c, so nix-build will build c.config.system.build.toplevel but it won't recursive into all of c.

dudebout commented 4 years ago

I see. Thanks for the explanation.