NixOS / nixops

NixOps is a tool for deploying to NixOS machines in a network or cloud.
https://nixos.org/nixops
GNU Lesser General Public License v3.0
1.82k stars 363 forks source link

Mac OS X deployment fails #1028

Open coretemp opened 5 years ago

coretemp commented 5 years ago

It is not possible to use NixOps (master) on Mac OS X to deploy something complex.

It's due to (see below for full log):

assertion failed at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/os-specific/linux/kernel/generic.nix:51:1

The manual says

NixOps runs on Linux and Mac OS X.

I don't think the manual is accurate given the above.

building all machine configurations...
error: while evaluating the attribute 'buildCommand' of the derivation 'nixops-machines' at /a/.nixpkgs-channels/pkgs/stdenv/generic/make-derivation.nix:148:11:
while evaluating anonymous function at /nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/share/nix/nixops/eval-machine-info.nix:369:46, called from /a/.nixpkgs-channels/lib/attrsets.nix:199:52:
while evaluating the attribute 'config.system.build.toplevel' at /a/.nixpkgs-channels/nixos/modules/system/activation/top-level.nix:263:5:
while evaluating 'foldr' at /a/.nixpkgs-channels/lib/lists.nix:34:20, called from /a/.nixpkgs-channels/nixos/modules/system/activation/top-level.nix:134:12:
while evaluating 'fold'' at /a/.nixpkgs-channels/lib/lists.nix:37:15, called from /a/.nixpkgs-channels/lib/lists.nix:41:8:
while evaluating 'showWarnings' at /a/.nixpkgs-channels/nixos/modules/system/activation/top-level.nix:100:18, called from /a/.nixpkgs-channels/nixos/modules/system/activation/top-level.nix:107:16:
while evaluating 'foldr' at /a/.nixpkgs-channels/lib/lists.nix:34:20, called from /a/.nixpkgs-channels/nixos/modules/system/activation/top-level.nix:100:23:
while evaluating 'fold'' at /a/.nixpkgs-channels/lib/lists.nix:37:15, called from /a/.nixpkgs-channels/lib/lists.nix:41:8:
while evaluating the attribute 'warnings' at /a/.nixpkgs-channels/lib/attrsets.nix:199:44:
while evaluating anonymous function at /a/.nixpkgs-channels/lib/modules.nix:75:45, called from /a/.nixpkgs-channels/lib/attrsets.nix:199:52:
while evaluating the attribute 'value' at /a/.nixpkgs-channels/lib/modules.nix:312:9:
while evaluating the option `warnings':
while evaluating the attribute 'isDefined' at /a/.nixpkgs-channels/lib/modules.nix:344:5:
while evaluating 'filterOverrides' at /a/.nixpkgs-channels/lib/modules.nix:419:21, called from /a/.nixpkgs-channels/lib/modules.nix:328:18:
while evaluating 'concatMap' at /a/.nixpkgs-channels/lib/lists.nix:102:18, called from /a/.nixpkgs-channels/lib/modules.nix:425:8:
while evaluating 'concatMap' at /a/.nixpkgs-channels/lib/lists.nix:102:18, called from /a/.nixpkgs-channels/lib/modules.nix:323:17:
while evaluating anonymous function at /a/.nixpkgs-channels/lib/modules.nix:323:28, called from undefined position:
while evaluating 'dischargeProperties' at /a/.nixpkgs-channels/lib/modules.nix:386:25, called from /a/.nixpkgs-channels/lib/modules.nix:324:62:
while evaluating the attribute 'value' at /a/.nixpkgs-channels/lib/modules.nix:216:48:
while evaluating the attribute 'config.warnings' at /a/.nixpkgs-channels/nixos/modules/system/boot/systemd.nix:702:5:
while evaluating 'mapAttrsToList' at /a/.nixpkgs-channels/lib/attrsets.nix:223:23, called from /a/.nixpkgs-channels/nixos/modules/system/boot/systemd.nix:702:29:
while evaluating the attribute 'services' at /a/.nixpkgs-channels/lib/attrsets.nix:199:44:
while evaluating anonymous function at /a/.nixpkgs-channels/lib/modules.nix:75:45, called from /a/.nixpkgs-channels/lib/attrsets.nix:199:52:
while evaluating the attribute 'value' at /a/.nixpkgs-channels/lib/modules.nix:312:9:
while evaluating the option `systemd.services':
while evaluating the attribute 'isDefined' at /a/.nixpkgs-channels/lib/modules.nix:344:5:
while evaluating 'filterOverrides' at /a/.nixpkgs-channels/lib/modules.nix:419:21, called from /a/.nixpkgs-channels/lib/modules.nix:328:18:
while evaluating 'concatMap' at /a/.nixpkgs-channels/lib/lists.nix:102:18, called from /a/.nixpkgs-channels/lib/modules.nix:425:8:
while evaluating 'concatMap' at /a/.nixpkgs-channels/lib/lists.nix:102:18, called from /a/.nixpkgs-channels/lib/modules.nix:323:17:
while evaluating anonymous function at /a/.nixpkgs-channels/lib/modules.nix:323:28, called from undefined position:
while evaluating 'dischargeProperties' at /a/.nixpkgs-channels/lib/modules.nix:386:25, called from /a/.nixpkgs-channels/lib/modules.nix:324:62:
while evaluating the attribute 'condition' at /a/.nixpkgs-channels/lib/modules.nix:467:14:
while evaluating the attribute 'condition' at /a/.nixpkgs-channels/lib/modules.nix:467:14:
while evaluating the attribute 'condition' at /a/.nixpkgs-channels/lib/modules.nix:467:14:
while evaluating the attribute 'services.klogd.enable' at /a/.nixpkgs-channels/lib/attrsets.nix:199:44:
while evaluating anonymous function at /a/.nixpkgs-channels/lib/modules.nix:75:45, called from /a/.nixpkgs-channels/lib/attrsets.nix:199:52:
while evaluating the attribute 'value' at /a/.nixpkgs-channels/lib/modules.nix:312:9:
while evaluating the option `services.klogd.enable':
while evaluating the attribute 'isDefined' at /a/.nixpkgs-channels/lib/modules.nix:344:5:
while evaluating anonymous function at /a/.nixpkgs-channels/lib/modules.nix:333:19, called from /a/.nixpkgs-channels/lib/modules.nix:333:14:
while evaluating the attribute 'value._type' at /a/.nixpkgs-channels/lib/modules.nix:424:73:
while evaluating the attribute 'value.content' at /a/.nixpkgs-channels/lib/modules.nix:482:14:
while evaluating the attribute 'default' at /a/.nixpkgs-channels/nixos/modules/services/logging/klogd.nix:12:7:
while evaluating 'versionOlder' at /a/.nixpkgs-channels/lib/strings.nix:359:22, called from /a/.nixpkgs-channels/nixos/modules/services/logging/klogd.nix:12:17:
while evaluating 'getVersion' at /a/.nixpkgs-channels/lib/strings.nix:383:16, called from /a/.nixpkgs-channels/nixos/modules/services/logging/klogd.nix:12:31:
while evaluating the attribute 'boot.kernelPackages.kernel' at /a/.nixpkgs-channels/nixos/modules/system/boot/kernel.nix:40:9:
while evaluating the attribute 'kernel.override' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/top-level/all-packages.nix:14086:12:
while evaluating the attribute 'linux_4_18' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/top-level/all-packages.nix:14016:3:
while evaluating 'callPackageWith' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:108:35, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/top-level/all-packages.nix:14016:16:
while evaluating 'makeOverridable' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:67:24, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:112:8:
while evaluating anonymous function at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/os-specific/linux/kernel/linux-4.18.nix:1:1, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:69:12:
while evaluating 'buildLinux' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/top-level/all-packages.nix:14321:16, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/os-specific/linux/kernel/linux-4.18.nix:5:1:
while evaluating 'callPackageWith' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:108:35, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/top-level/all-packages.nix:14321:23:
while evaluating 'makeOverridable' at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:67:24, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:112:8:
while evaluating anonymous function at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/os-specific/linux/kernel/generic.nix:1:1, called from /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/lib/customisation.nix:69:12:
assertion failed at /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/os-specific/linux/kernel/generic.nix:51:1
Traceback (most recent call last):
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/bin/.nixops-wrapped", line 990, in <module>
    args.op()
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/bin/.nixops-wrapped", line 411, in op_deploy
    max_concurrent_activate=args.max_concurrent_activate)
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/lib/python2.7/site-packages/nixops/deployment.py", line 1056, in deploy
    self.run_with_notify('deploy', lambda: self._deploy(**kwargs))
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/lib/python2.7/site-packages/nixops/deployment.py", line 1045, in run_with_notify
    f()
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/lib/python2.7/site-packages/nixops/deployment.py", line 1056, in <lambda>
    self.run_with_notify('deploy', lambda: self._deploy(**kwargs))
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/lib/python2.7/site-packages/nixops/deployment.py", line 996, in _deploy
    self.configs_path = self.build_configs(dry_run=dry_run, repair=repair, include=include, exclude=exclude)
  File "/nix/store/9v1hri5laa7dd54lz5k33dnnl5pmrz7r-nixops-1.6.1pre0_abcdef/lib/python2.7/site-packages/nixops/deployment.py", line 664, in build_configs
    raise Exception("unable to build all machine configurations")
Exception: unable to build all machine configurations
LisannaAtHome commented 5 years ago

Are you setting the configuration option nixpkgs.system = "x86_64-linux"? Are you ever importing <nixpkgs> directly?

coretemp commented 5 years ago

No, for the first question in the deployed configurations.

I do have lines of the form foo = import <nixpkgs {};, if that answers your second question.

LisannaAtHome commented 5 years ago

In your case (and mine!), evaluation happens on darwin and the build happens on linux. That assertion that's failing in /nix/store/n3k7w0vjbwhnnzsmnl6blh3r83gciabq-source/pkgs/os-specific/linux/kernel/generic.nix:51:1 means that you didn't tell nixpkgs what system it should build for, and so it assumed the evaluation host (darwin).

To fix this, pass in localSystem.system = "x86_64-linux"; to any manually-imported <nixpkgs> spots, and set the nixpkgs.localSystem NixOS option in your machine's configuration to the same.

coretemp commented 5 years ago

Please clarify the part of your sentence with "your machine's configuration". My current way of reading this is, that it's about the machines being deployed.

LisannaAtHome commented 5 years ago

Your machine's configuration = the NixOS configuration for the machine that you're specifying in NixOps.

coretemp commented 5 years ago

Is there any way I can get the line number of the import <nixpkgs> {} that causes this?

(I consider getting an answer to this question less important. )

coretemp commented 5 years ago

@ledettwy I made some progress, but now I hit the following:

This was a change that was put in place to have a recent version of Nix in place. I would like to continue to use an arbitrary version of Nix as opposed to only released versions for flexibility.

a 'x86_64-linux' is required to build '/nix/store/i5ssk4caj5gzyqgx5ap4g79pihlscapd-source.drv', but I am a 'x86_64-darwin'

The line it refers to is the one defining the nix attribute.

tools.nix:
{ rev, sha256 }:

let bootstrap = import <nixpkgs> { localSystem.system = "x86_64-linux";};
in
bootstrap.fetchFromGitHub {
    owner = "NixOS";
    repo  = "nixpkgs";
    inherit rev sha256;
}
{ config, lib, pkgs, ... }:

let
  recent_nixpkgs = (import ./tools.nix)
    {rev = "dd57a1fd402125727eaa88019b70bfb82851b05a";
     sha256 = "0kmbslgrwb66mjmssphajd4ipx0hwhggs3qnrc7d4nhq0n77zpw6";
    };
nixpkgs_with_nix_2dot1 = import recent_nixpkgs { localSystem.system = "x86_64-linux";};
in
{
  options = {};
  config = {
    nixpkgs.localSystem.system = "x86_64-linux";
    nixpkgs.config.packageOverrides = pkgs: {
        nix = nixpkgs_with_nix_2dot1.nixUnstable.overrideAttrs (oldAttrs: {
          src = nixpkgs_with_nix_2dot1.fetchFromGitHub {
            owner = "NixOS";
            repo = "nix";
            rev = "2ca6ef0fec150209994814fada3533fbd0f9f764";
            sha256 = "0yfs17f0gac2yzrz4zkkwh132lbwkyzg60dxyx44bzqynh2ibky0";
        };
        });};
  };
}
kirelagin commented 4 years ago

This is only a problem with libvirt. You have not shown your deployment description, but I suspect that is the culprit. If you change the deployment target to, say, VirtualBox, it will, most likely, work.

kirelagin commented 4 years ago

NixOS/nixpkgs#73470