nix-community / terraform-nixos

A set of Terraform modules that are designed to deploy NixOS [maintainer=@adrian-gierakowski]
Apache License 2.0
333 stars 61 forks source link

Copy .nix files to /etc/nixos as well? #39

Open fiksn opened 3 years ago

fiksn commented 3 years ago

First apologies for asking a noob question. I am playing around with deploy_nixos w/ DigitalOcean droplets and everything works great so far. I use a templatefile to generate configuration.nix (for IPs and stuff) and have some imports ala

 imports = [
    <nixpkgs/nixos/modules/virtualisation/digital-ocean-image.nix>
    ./common.nix
    ./private.nix
  ]

Now what I would like to achieve is beside applying the configuration also copying the closure of all nix files to /etc/nixos on the machine somehow (you might imagine that also common.nix imports some blahblah/one.nix and that should become /etc/nixos/blahblah/one.nix then and so on). The server is converted through nixos-infect from Ubuntu 20.04 so in /etc/nixos there is still the stuff left from nixos-infect. Of course the usually way should be terraform apply from my workstation but I am worried that I might call nixos-rebuild switch by mistake directly on the server which could lead to problems. Or should I just rm -rf /etc/nixos on the server?

fiksn commented 3 years ago

Some sort of find / scp / rsync combo is not good , because I also have some tests.nix that is is not referenced through configuration.nix. And then digital-ocean-image.nix is also something that I don't need to copy (since it is already there).

zimbatm commented 3 years ago

The best option is to delete /etc/nixos on the target machine. You can do that by using the "remote-exec" provisioner, or by adding it in a systemd unit in the machine configuration.nix.

It would be nice if it was possible to copy all the nix files onto the server, so that it can be rebuilt manually, but that's quite difficult to achieve. To reproduce the config, deploy_nixos would have to capture not only the configuration.nix, but also be able to follow all the imported modules. And which version of nixpkgs is being used.

fiksn commented 3 years ago

I came up with this "contraption" 🙂

{ pkgs ? import <nixpkgs> {}, ... }:
let
  lib = pkgs.lib;

  resolveOne = file: let p = pkgs.callPackage file {}; in lib.filter (x: !lib.hasPrefix "/nix" (builtins.toPath x)) (if lib.hasAttr "imports" p then p.imports else []);
  oneStep = result: lib.unique (lib.foldl' (x: y:  x ++ resolveOne y) result result);
  resolve = input: let resolveRec = x: let y = oneStep x; in if lib.length x == lib.length y then y else resolveRec y; in resolveRec [ input ];

  getPrefix = main: let file = builtins.toPath main; name = builtins.baseNameOf file; root = lib.replaceStrings [ "/${name}" ] [ ""] file; in root;
  getRelativePaths = main: let prefix = getPrefix main; in map (x: lib.replaceStrings [ prefix ] [ "." ] (builtins.toPath x)) (resolve main);
  getRelativePathsStr = main: builtins.concatStringsSep "\n" (getRelativePaths main);
in
pkgs.mkShell {
  shellHook = ''
    echo "ssh machine rm -rf /etc/nixos"
    echo "${getRelativePathsStr ./configuration.nix}" | xargs -n 1 -I {} echo "ssh machine mkdir -p /etc/nixos/$(pathname {} 2>/dev/null) ; scp {} machine:/etc/nixos/{}"
  '';
}

Obviously this is just an approximation, since nixpkgs might be different on remote machine.

zimbatm commented 3 years ago

:see_no_evil:

Another thing you can do is set nix.nixPath = [ "nixpkgs=${pkgs}" ]; so nixpkgs also get pushed and set to NIX_PATH.

zimbatm commented 3 years ago

There is also https://search.nixos.org/options?channel=20.09&show=system.copySystemConfiguration&from=0&size=30&sort=relevance&query=copy , but that only works if the config is self-contained into a single file.