Closed dhess closed 7 years ago
You don't need the nix-daemon for remote builds, does a regular build work?
$ nix-build '<nixpkgs>' -A hello --argstr system x86_64-linux
$ nix-build '<nixpkgs>' -A hello --argstr system x86_64-linux --check
The former does (but it just downloads the nar file from the binary cache); the latter doesn't:
% nix-build '<nixpkgs>' -A hello --argstr system x86_64-linux --check
fetching path ‘/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz’...
*** Downloading ‘https://cache.nixos.org/nar/0yc68z03plkyms2vgr4f8byya75zcgqzbqsmbvazv8cf1fi70vdz.nar.xz’ to ‘/nix/store/3x7dwzq014bblazs7kq20p9hyzz0qh8g-hello-2.10.tar.gz’...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 709k 100 709k 0 0 488k 0 0:00:01 0:00:01 --:--:-- 489k
...
*** Downloading ‘https://cache.nixos.org/nar/0rd1p8zicpr97wnb11l0nab19130prdjnhz2xdcc19igkf16hi53.nar.xz’ to ‘/nix/store/aqafkcxpw23kk9fp2bavyczck4nnh9m1-stdenv’...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 8124 100 8124 0 0 11261 0 --:--:-- --:--:-- --:--:-- 15836
checking path(s) ‘/nix/store/7bqd18jfn63pqkcs78lcas5mcp10vmpy-hello-2.10’
error: a ‘x86_64-linux’ is required to build ‘/nix/store/p1kilmvgc7bjk2b03958mbj17qbcirqp-hello-2.10.drv’, but I am a ‘x86_64-darwin’
Here are the relevant environment variables:
NIX_BUILD_HOOK=/Users/dhess/.nix-profile/libexec/nix/build-remote.pl
NIX_CONF_DIR=/Users/dhess/.nixconf
NIX_REMOTE_SYSTEMS=/Users/dhess/.nixconf/remote-systems.conf
NIX_PATH=nixpkgs=/Users/dhess/git/nixpkgs
NIX_CURRENT_LOAD=/tmp/nix/current-load
My slightly modified version of remote-build-env
looks like this:
#!/usr/bin/env bash
mkdir -p /tmp/nix/current-load
chmod a+rwX /tmp/nix/current-load
export NIX_BUILD_HOOK="${HOME}/.nix-profile/libexec/nix/build-remote.pl"
export NIX_REMOTE_SYSTEMS="${NIX_CONF_DIR}/remote-systems.conf"
export NIX_CURRENT_LOAD="/tmp/nix/current-load"
My nixpkgs
repo is up-to-date with https://github.com/NixOS/nixpkgs/commit/13d6681ce72ac265f03013247f60ab3d001779e0 and my nix-env
version is 1.11.6
.
Thanks for the assistance!
Here is the verbose output from the failing command: https://gist.github.com/dhess/465e84c60e517ca98252870bfdd62a29
Where did you put the signing keys? If you set NIX_CONF_DIR they might also need to be in there.
The signing keys are also in NIX_CONF_DIR
.
@LnL7 are you able to get the ssh remote builder to work as expected on your Mac, running your nix-darwin
environment?
Figuring out why it doesn't work is quite tricky, there are a number of things that fail silently.
I did just notice that you seem to be using your public key in the remote-systems.conf
, I think that might be the problem here.
Hmm. I did replace the key on the ssh container with my own per the instructions in this repo's README.md, and I can ssh into the container as root using that key.
Yes, but the build hook uses the private keys specified in NIX_REMOTE_SYSTEMS
while you configured it with the .pub
file. I expect that causes the builder to be ignored silently.
Good catch! Unfortunately, after fixing remote-systems.conf
to use my private key, I'm still getting the same failures.
I think it's pretty likely that this is some weird issue with nix-build
and/or my environment, and not the ssh Docker image. You can close this issue if you like. Otherwise, I'll report back here when I figure out what's going on. I would expect that at most a change is needed to the README.
Perhaps https://github.com/LnL7/nix-docker/pull/5 helps?
I've got the ssh remote builder container running in Docker on my macOS box. I can ssh into the container from the macOS host, and I've created and uploaded a signing key for remote builds.
However, when trying to use the container from macOS as a remote builder for Linux targets, I'm running into
nixpkgs
assertions at various places. For example, when I attempt to build one of my Haskell projects for Linux from macOS, using the remote builder container, after sourcing yourremote-build-env
script, I get the following:I'm using
nixpkgs
from git; the line shown in the error message above refers to the cross-compiling check:I'm having a similar problem trying to use the ssh container with
nixops
to provision a NixOS physical host from macOS.nixops
breaks as soon as it tries to build a kernel for the target host, though at least in this case it's finding my localremote-systems.conf
file:Is a local
nix-daemon
a pre-requisite for using a remote builder? I'm not running one at the moment as I'm not using yournix-darwin
repo and I've never needed it for my macOS work.My
remote-systems.conf
file looks like this: