LXC image is unable to `nixos-rebuild switch`

asbachb commented 5 years ago

Summary nixos-generate generated image is not able to perform nixos-rebuild switch.

Host

  virtualisation.lxc = {
    enable = true;
    lxcfs.enable = true;
    defaultConfig = "lxc.include = ${pkgs.lxcfs}/share/lxc/config/common.conf.d/00-lxcfs.conf";
  };

  virtualisation.lxd.enable = true;

Guest

[root@nixos:~]# cat /etc/nixos/configuration.nix
{ config, pkgs, ... }:

{
  imports = [  ];

  boot.isContainer = true;

  systemd.services."getty@".enable = false;
}

[root@nixos:~]# nixos-rebuild switch
building Nix...
building the system configuration...
activating the configuration...
setting up /etc...
mount: /dev: cannot remount devtmpfs read-write, is write-protected.
mount: /dev/pts: cannot remount devpts read-write, is write-protected.
mount: /dev/shm: cannot remount tmpfs read-write, is write-protected.
mount: /proc: cannot remount proc read-write, is write-protected.
mount: /run: cannot remount tmpfs read-write, is write-protected.
mount: /run/keys: cannot remount ramfs read-write, is write-protected.
mount: /run/wrappers: cannot remount tmpfs read-write, is write-protected.
Activation script snippet 'specialfs' failed (32)
reloading user units for root...
setting up tmpfiles
warning: the following units failed: sys-kernel-debug.mount

● sys-kernel-debug.mount - Kernel Debug File System
   Loaded: loaded (/nix/store/jabq4nrn21fm49zvaqfpg9986xsr8g5r-systemd-239.20190219/example/systemd/system/sys-kernel-debug.mount; enabled; vendor preset: enabled)
   Active: failed (Result: exit-code) since Thu 2019-09-12 16:51:46 UTC; 28ms ago
    Where: /sys/kernel/debug
     What: debugfs
     Docs: https://www.kernel.org/doc/Documentation/filesystems/debugfs.txt
           https://www.freedesktop.org/wiki/Software/systemd/APIFileSystems

Sep 12 16:51:46 nixos systemd[1]: Mounting Kernel Debug File System...
Sep 12 16:51:46 nixos mount[3231]: mount: /sys/kernel/debug: permission denied.
Sep 12 16:51:46 nixos systemd[1]: sys-kernel-debug.mount: Mount process exited, code=exited status=32
Sep 12 16:51:46 nixos systemd[1]: sys-kernel-debug.mount: Failed with result 'exit-code'.
Sep 12 16:51:46 nixos systemd[1]: Failed to mount Kernel Debug File System.
warning: error(s) occurred while switching to the new configuration

Maybe contains relevant information: https://github.com/NixOS/nixpkgs/issues/9735#issuecomment-500164017

zimbatm commented 5 years ago

Yeah it seems more related to a nixpkgs + LXC issue. This project just takes the nixos profiles and creates a generic interface to produce the images, with a bit of nix glue code to make that work.

n8henrie commented 2 years ago

I was able to get this working with lxc image import $(nixos-generate -f lxc-metadata) $(nixos-generate -f lxc)^2 followed by lxc launch -c security.nesting=true 123 nixos^0 where 123 was the imported image fingerprint.

It looks like it should also be possible without security.nesting with some lxc profile settings^1.

Probably unsurprising to experienced nix users, I had to set boot.isContainer = true; (thanks to this thread) before I could nixos-rebuild switch due to errors about filesystems and grub.

nix-community / nixos-generators

LXC image is unable to `nixos-rebuild switch` #41