nix-community / raspberry-pi-nix

NixOS modules to aid in configuring NixOS for raspberry pi products
MIT License
144 stars 34 forks source link

set `boot.initrd.systemd.enable = false` when uboot is disabled #24

Closed madprogrammer closed 1 month ago

madprogrammer commented 2 months ago

Hello!

Not sure what I am doing wrong. Trying to generate a SD card image from my config. The problem is that the resulting SD image only contains /sbin/init and /nix/store (see screenshot). According to the source code of this flake, this seems to be intended behaviour, however, the system doesn't boot - it complains about /proc, /sys, /dev missing and can't mount the relevant filesystems. Also, root=fstab is added to cmdline.txt in addition to the correct root=PARTUUID=2178694e-02. I believe it is intended for systemd running in an initrd, however there seems to be no initrd because if I keep this option the kernel tries to mount rootfs from fstab device which is obviously not available, and panics. Could you please give any insights on what could be wrong? Thanks!

{ pkgs, inputs, ... }: {
  imports = [
    inputs.raspberry-pi-nix.nixosModules.raspberry-pi
    ../common/global
    ../common/users/sergey
  ];

  networking = {
    hostName = "rpi5";
    networkmanager.enable = true;
  };

  time.timeZone = "Europe/Amsterdam";
  users.users.root.initialPassword = "root";
  boot.initrd.systemd.enableTpm2 = false;

  # libcamera build fails if this is enabled
  raspberry-pi-nix.libcamera-overlay.enable = false;

  hardware.raspberry-pi = {
    config = {
      all = {
        options = {
          usb_max_current_enable = {
            enable = true;
            value = true;
          };
        };
        dt-overlays = {
          # hack to generate dtparam=nvme
          vc4-kms-v3d = {
            enable = true;
            params = { nvme = { enable = true; }; };
          };
        };
        base-dt-params = {
          pciex1_1 = {
            enable = true;
            value = "gen3";
          };
        };
      };
    };
  };

  i18n = {
    defaultLocale = "en_US.UTF-8";
  };

  services.udisks2.enable = true;

  environment.systemPackages = with pkgs; [
    wget
  ];

  system.stateVersion = "24.05";
}

image

SebTM commented 2 months ago

Hmm, not sure I understand the issue - can you share the whole code (with flake) or how do you build the image?

madprogrammer commented 2 months ago

It is a part of my large flake with multiple configurations for multiple hosts, so I would need to isolate it into a separate flake to be able to share. I build the image with the following command: nix build .#nixosConfigurations.rpi5.config.system.build.sdImage

The flake is here. The code above is rpi5.nix (referenced by the flake in nixosConfigurations).

I am getting an image successfully built which contains the boot partition with kernel.img, config.txt and other files, and NIXOS_SD partition with the root file system. When I write the image onto NVMe and try to boot, the kernel starts to boot but then panics after a short while because it tries to mount root=fstab; here's my cmdline.txt generated by the flake: console=serial0,115200n8 console=tty1 root=PARTUUID=2178694e-02 rootfstype=ext4 fsck.repair=yes rootwait init=/sbin/init root=fstab loglevel=7. If I remove the root=fstab argument and only keep the root=PARTUUID=2178694e-02 one, then it continues to boot but again crashes when it tries to mount /proc, /dev, /sys etc. filesystems, because there are no such mount points on the NIXOS_SD partition.

I suspect that this behavior is due to the kernel boots without an initrd, however I am not sure. It seems that disabling uboot has become default only a week ago, and I am not sure if it's supposed to work yet. U-Boot doesn't support booting from NVMe so I can't use it. The image generated for U-Boot, however, does have an initrd.

Hope I made it somewhat more clear, sorry for the confusion.

madprogrammer commented 2 months ago

Ok, I figured it out, it was due to boot.initrd.systemd.enable = mkDefault true set in one of my included configuration files. Set it to false and it booted and the root=fstab option was also gone from cmdline.txt.

tstat commented 2 months ago

Thanks for reporting. It is normal to for that partition to contain only /nix and /sbin. /proc, /dev, and /sys are created by the init script, home directories and etc are created by activation scripts. When uboot is false we don't use an initrd.

This issue is still relevant though, we should set boot.initrd.systemd.enable = false if uboot is disabled.

tstat commented 1 month ago

On second thought, I'm not sure this is something raspberry-pi-nix should be concerned with.