kernel-less profile/module in nixpkgs/nixos?

Mic92 commented 6 years ago

We already have a isContainer flag. Now there is an open pull request to add similar option for paravirtualized kernels. I think we could need some module, we can include in case we don't want nixos to handle the kernel/bootloader and instead only provide an init script.

I could imagine the following use cases:

containers
paravirtualized xen guest
some providers that provide their own kernels (i.e. scaleway)
some ugly embedded hardware, where we use the non-upstream kernel/u-boot provided by the vendor and only manage the nixos userland.

Any suggestions? cc @dezgeg @andir @NeQuissimus

markuskowa commented 6 years ago

That would simplify the integration of some less popular ARM boards.

NeQuissimus commented 6 years ago

I can see the merit but not sure how exactly it would look.

angerman commented 6 years ago

If there is an idea how this could be done properly, I'd be happy to take a shot at it.

Mic92 commented 6 years ago

I would say propose the following. We could start small and keep the other occurrences of isContainer:

# nixos/modules/profiles/no-kernel.nix
{
   boot.loader.initScript = true;

   ## the following options must be added:
   # affects nixos/modules/system/activation/top-level.nix 
   # and nixos/modules/system/boot/modprobe.nix
   # and nixos/modules/system/boot/stage-1.nix
   # -> this should replace the existing isContainer usage.
   system.installKernel = true;
   # nixos/modules/system/boot/loader/grub/grub.nix
   boot.loader.enable = true;

   ## instead of using installKernel in nixos/modules/system/boot/stage-1.nix, 
   # there could be a `boot.initrd.enable` for cases where can provide a initrd but no kernel - is there a use case that?
   #boot.initrd.enable = true;
}

SLNOBody commented 6 years ago

If there is an idea how this could be done properly, I'd be happy to take a shot at it.

Simple: completely separate bootloader profile generation from init profile generation as GuixSD and SLNOS do.

TBH, I find the current way NixOS does bootloaders (and, as consequence: the lack of proper PXE booting I implemented over changes rejected in NixOS somewhere around #2920, proper live ISOs discussed on "nix-devel" on 2014-02-19, and this issue) strange, to say the least.

"Did you know, that every NixOS profile carries a script that is ought to be able to parse all future versions of all other NixOS profiles to generate GRUB menus for them?" "Eh? I'm pretty sure that is impossible." "Moreover, did you know that those scripts run on configuration switches, not on profile updates. I.e. that your GRUB menu is almost always stale?" "Why haven't anybody fixed that already?" "Because @edolstra said it was okay the way it is." (See conversation between me and @edolstra on "nix-devel" dated 2014-02-19..2014-02-25.)

Such is life.

oxij commented 6 years ago

If there is an idea how this could be done properly, I'd be happy to take a shot at it.

Simple: completely separate bootloader profile generation from init profile generation as GuixSD and SLNOS do.

TBH, I find the current way NixOS does bootloaders (and, as consequence: the lack of proper PXE booting I implemented over changes rejected in NixOS somewhere around #2920, proper live ISOs discussed on "nix-devel" on 2014-02-19, and this issue) strange, to say the least.

"Did you know, that every NixOS profile carries a script that is ought to be able to parse all future versions of all other NixOS profiles to generate GRUB menus for them?" "Eh? I'm pretty sure that is impossible." "Moreover, did you know that those scripts run on configuration switches, not on profile updates. I.e. that your GRUB menu is almost always stale?" "Why haven't anybody fixed that already?" "Because @edolstra said it was okay the way it is." (See conversation between me and @edolstra on "nix-devel" dated 2014-02-19..2014-02-25.)

Such is life.

Mic92 commented 6 years ago

@oxij do you have to then choose after booting into the boot loader entry a second time to select the init profile?

oxij commented 6 years ago

@oxij do you have to then choose after booting into the boot loader a second time to select the init?

Well, you could implement that, but ATM neither Guix nor SLNOS do such things. I find the idea somewhat crazy.

My point is that boot menu generator should be a separate tool, not a nixos module.

nixos module should simply dump a versioned JSON/S-expr/XML/whatever description of the boot configuration in profile root,
update-grub tool should be mostly independent tool that parses those dumps, generate menus and updates GRUB.

In SLNOS we do a kind of MetaNixOS approach to this (because imagine having to work with two slightly different versions of NixOS --- upstream and SLNOS --- at the same time). Instead of having nixos-rebuild do all the things NixOS's nixos-rebuild does and break the bootability of its slightly different sibling by regenerating the wrong boot menu entry, we make nixos-rebuild do only the rebuild. The rest of it is handled with a separate tool that knows about both ways of doing boot things.

I think that is actually the proper way to do things even if you don't want to run slightly different siblings, because if you rewrite everything boot-related to be this way you can then, for instance, pack two different OSes on a single ISO with a common boot menu (* not yet in SLNOS), properly kexec between different OSes, generate properly updating PXE menus from profiles that change frequently, etc.

7c6f434c commented 6 years ago

Instead of having nixos-rebuild do all the things NixOS's nixos-rebuild does and break the bootability of its slightly different sibling by regenerating the wrong boot menu entry, we make nixos-rebuild do only the rebuild. The rest of it is handled with a separate tool that knows about both ways of doing boot things.

Does it allow different systems to be in different profiles (i.e. not system)? Is it easy to use SLNOS with Nixpkgs master for simple packages? (Alternatively: will patches for that be accepted?) Because I run a system of custom bootscripts, sinit and really weird way of launching stuff like agetty; I want Nixpkgs master branch for packages, and I want stuff like CUPS launcher from something Nix-based, and SLNOS sounds like a better service-set upstream for my needs.

oxij commented 6 years ago

Does it allow different systems to be in different profiles (i.e. not system)?

Yes.

Is it easy to use SLNOS with Nixpkgs master for simple packages? (Alternatively: will patches for that be accepted?)

Well, SLNOS/pkgs is mostly nixpkgs with heavy use of use-flags (#12877, see my comments there, the impl is slightly different now, though) and occasional bunch of patches from Triton. So, probably. When SLNOS/pkgs needs something essential to keep being compatible with nixpkgs (like #33057) I try really hard to push it in (thanks @vcunat, again).

Because I run a system of custom bootscripts, sinit and really weird way of launching stuff like agetty;

Right. We stole most of this part of SLNOS from your mails to "nix-devel", to be honest.

I want Nixpkgs master branch for packages, and I want stuff like CUPS launcher from something Nix-based, and SLNOS sounds like a better service-set upstream for my needs.

Honestly, its not. Yet. But we are working on it and that part is the part we plan to make public first unless nixpkgs rejects something very essential to SLNOS that would make it very hard to merge them in the meantime.

That said, nixpkgs being BSD (i.e. lack of pressure to publish modifications), our internal policies, and my lack of time make it very hard to push SLNOS patches into public --- the fact which I regret daily.

7c6f434c commented 6 years ago

We stole most of this part of SLNOS from your mails to "nix-devel"

It is nice that they were useful for you! Note: I probably have an even more weird version right now. My next step is also to finally get fast and non-annoying fully atomic switches (which means read-only /etc but I do want to have mutable users… it's complicated but fits OK with what I am actually trying to build).

SLNOS sounds like a better service-set upstream for my needs Honestly, its not. Yet.

My wishes: understanding that the current choice of init system is a historical accident, some CUPS support, preferably being able to have both source system and what I actually want to run installed at once.

Code quality? Well, if X.org and CUPS start at all, I guess I can compose something out of all that.

If too much incapsulation gets enforced, then I might have some problems…

make it very hard to push SLNOS patches into public --- the fact which I regret daily

Well, if I start using SLNOS as a service upstream (but I want fresh stuff more than use flags), at least I would have an incentive to push critical-but-safe parts. And I will have slightly lower friction for that .I find it unlikely that all my desires fit perfectly with SLNOS goals, so I will not fully switch. That means I will commit various package updates to Nixpkgs anyway and I will be able to avoid disclosure of SLNOS secrets by ignorance instead of policy.

Mic92 commented 6 years ago

Could you please not use this issue as a thread for off-topic discussions?

stale[bot] commented 4 years ago

Thank you for your contributions.

This has been automatically marked as stale because it has had no activity for 180 days.

If this is still important to you, we ask that you leave a comment below. Your comment can be as simple as "still important to me". This lets people see that at least one person still cares about this. Someone will have to do this at most twice a year if there is no other activity.

Here are suggestions that might help resolve this more quickly:

Search for maintainers and people that previously touched the related code and @ mention them in a comment.
Ask on the NixOS Discourse.
Ask on the #nixos channel on irc.freenode.net.

samueldr commented 4 years ago

Note that since then, boot.initrd.enable was added (#80114).

It is not a replacement for the feature, but nonetheless useful in similar circumstances.

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

Mic92 commented 3 years ago

For VM images I use the following NixOS configuration to have not kernels installed. This is useful for kernel development.

# $ nix-build minimal-vm.nix
# $ cp ./result/nixos.qcow2 . && chmod +w nixos.qcow2
# $ qemu-system-x86_64 -s -S  \
#   -kernel arch/x86/boot/bzImage \
#   -hda nixos.qcow2 \
#   -append "root=/dev/sda console=ttyS0 nokalsr" \
#   -nographic -enable-kvm
{ pkgs ? import <nixpkgs> {} }:

import (pkgs.path + "/nixos/lib/make-disk-image.nix") {
  inherit pkgs;
  inherit (pkgs) lib;
  config = (import (pkgs.path + "/nixos/lib/eval-config.nix") {
    inherit (pkgs) system;
    modules = [{
      imports = [
        (pkgs.path + "/nixos/modules/profiles/qemu-guest.nix")
      ];
      boot.loader.grub.enable = false;
      boot.initrd.enable = false;
      boot.isContainer = true;
      boot.loader.initScript.enable = true;
      # login with empty password
      users.extraUsers.root.initialHashedPassword = "";

      services.mingetty.helpLine = ''
        Log in as "root" with an empty password.
        If you are connect via serial console:
        Type Ctrl-a c to switch to the qemu console
        and `quit` to stop the VM.
      '';
      documentation.doc.enable = false;
    }];
  }).config;
  partitionTableType = "none";
  diskSize = 8192;
  format = "qcow2";
}

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

samueldr commented 3 years ago

Still would be desirable.

E.g. with Mobile NixOS we want to be able to manage the kernel ourselves, since the NixOS tooling assumes a lot (rightfully so) about the kernel. Being able to opt-out as a whole from those assumptions would increase the composability of NixOS as a whole.

Note that it could be an internal option, with the express drawback that this understandably breaks some part of the assumptions. As long as it introduces a method to drop the "NixOS managed" kernel from the configuration.

NixOS / nixpkgs

kernel-less profile/module in nixpkgs/nixos? #33940