nix-community / disko

Declarative disk partitioning and formatting using nix [maintainers=@Lassulus @Enzime @iFreilicht]
MIT License
1.84k stars 195 forks source link

What's wrong with the config below? #354

Closed nikolay closed 3 weeks ago

nikolay commented 1 year ago

I get this error:

image

With this config:

{
  disks,
  lib,
  ...
}: {
  disko.devices = {
    disk = lib.genAttrs disks (device: {
      type = "disk";
      name = lib.removePrefix "_" (builtins.replaceStrings ["/"] ["_"] device);
      device = device;
      content = {
        type = "gpt";
        partitions =
          (
            lib.mkIf (device == (builtins.elemAt disks 0)) {
              esp = {
                type = "EF00";
                size = "512M";
                content = {
                  type = "filesystem";
                  format = "vfat";
                  mountpoint = "/boot";
                };
              };
            }
          )
          // {
            zfs = {
              size = "100%";
              content = {
                type = "zfs";
                pool = "zroot";
              };
            };
          };
      };
    });
    zpool = {
      zroot = {
        type = "zpool";
        mode = "raidz";
        rootFsOptions = {
          compression = "zstd";
          "com.sun:auto-snapshot" = "false";
        };
        datasets = {
          "data" = {
            type = "zfs_fs";
            options.mountpoint = "none";
          };
          "ROOT" = {
            type = "zfs_fs";
            options.mountpoint = "none";
          };
          "ROOT/empty" = {
            type = "zfs_fs";
            mountpoint = "/";
            options.mountpoint = "legacy";
            postCreateHook = ''
              zfs snapshot zroot/ROOT/empty@start
            '';
          };
          "ROOT/nix" = {
            type = "zfs_fs";
            mountpoint = "/nix";
            options.mountpoint = "legacy";
          };
        };
      };
    };
  };
}
nikolay commented 1 year ago

The issue was my deployment script. Sorry.

nikolay commented 1 year ago

Never mind, it wasn't the deployment script.

phaer commented 1 year ago
        lib.mkIf (device == (builtins.elemAt disks 0)) {

I believe you can just use if (device == (builtins.elemAt disks 0)) then ... else ....

nikolay commented 1 year ago

@phaer Yeah, sorry. My last version, which works, is this one:

{
  disks ? ["/dev/sda" "/dev/sdb" "/dev/sdc"],
  lib,
  ...
}: let
  zfs = {
    type = "zfs";
    pool = "zroot";
  };
in {
  disko.devices = {
    disk = lib.genAttrs disks (device: {
      type = "disk";
      name = lib.removePrefix "_" (builtins.replaceStrings ["/"] ["_"] device);
      device = device;
      content =
        if (device == (builtins.elemAt disks 0))
        then {
          type = "gpt";
          partitions = {
            esp = {
              type = "EF00";
              size = "512M";
              content = {
                type = "filesystem";
                format = "vfat";
                mountpoint = "/boot";
              };
            };
            zfs = {
              size = "100%";
              content = zfs;
            };
          };
        }
        else zfs;
    });
    zpool = {
      zroot = {
        type = "zpool";
        mode = "raidz";
        rootFsOptions = {
          compression = "zstd";
          "com.sun:auto-snapshot" = "false";
        };
        mountpoint = "/";
        postCreateHook = "zfs snapshot zroot@blank";
        datasets = {
          "root" = {
            type = "zfs_fs";
            options.mountpoint = "none";
          };
        };
      };
    };
  };
}

So, I guess, some of the examples don't really work, i.e. those having partitions with just zfs.

Yet, not the issue is that bootctl fails:

image

phaer commented 1 year ago

So, I guess, some of the examples don't really work, i.e. those having partitions with just zfs.

Did you test the examples to verify? If so, can you post a log? :)

Yet, not the issue is that bootctl fails:

It says that /boot does not exist - did you check the log above whether it actually gets created and mounted? You should see creation and mount commands with --debug.

nikolay commented 1 year ago

@phaer It wasn't getting created, and I'm not sure how I've fixed it, but it works now. All my code is here: https://github.com/nikolay/zfs-on-nix-test

I'm basically trying to run NixOS on Scaleway Dedibox, but I'm a total noob when it comes to Nix, NixOS, and ZFS.

Now, the error is different:

image

nikolay commented 1 year ago

@phaer Unfortunately, /boot randomly does not mount during the installation. 😒 The second time I ran the script, it failed again. I am a little concerned about nixos-install not failing when a critical part fails.

iFreilicht commented 3 weeks ago

For posterity, I believe the issue is how you define your zpool:

    zpool = {
      zroot = {
        type = "zpool";
        mode = "raidz";
        rootFsOptions = {
          compression = "zstd";
          "com.sun:auto-snapshot" = "false";
        };
        mountpoint = "/";
        postCreateHook = "zfs snapshot zroot@blank";
        datasets = {
          "root" = {
            type = "zfs_fs";
            options.mountpoint = "none";
          };
        };
      };

You shouldn't mount the rootFS at /, that's what the root dataset is for. Instead, try:

    zpool = {
      zroot = {
        type = "zpool";
        mode = "raidz";
        rootFsOptions = {
          compression = "zstd";
          "com.sun:auto-snapshot" = "false";
        };
        postCreateHook = "zfs snapshot zroot@blank";
        datasets = {
          "root" = {
            type = "zfs_fs";
            options.mountpoint = "/";
          };
        };
      };

I'm assuming that you've either solved this issue since last year or have given up, so I'll close this for now to keep our board tidy. However, if you're still working on this and this couldn't solve your issue, feel free to re-open it!