search

nix-community / disko

Declarative disk partitioning and formatting using nix [maintainers=@Lassulus @Enzime @iFreilicht]
MIT License
1.75k stars 192 forks source link

Can't automount ZFS dataset without using mountpoint="legacy" #581

Open Toomoch opened 6 months ago

Toomoch commented 6 months ago

First, I created the following zpool:

    zpool = {
      zstorage = {
        type = "zpool";
        mode = "mirror";
        rootFsOptions = {
          compression = "zstd";
        };
        mountpoint = "/zstorage";

        datasets = {
          data = {
            type = "zfs_fs";
            mountpoint = "/zstorage/data";
            options = {
            };
          };
          share = {
            type = "zfs_fs";
            mountpoint = "/zstorage/share";
            options = {
            };
          };
        };
      };
    };

Then ran the vm-test, and it gives the following error when booting:

vm-test-run-disko-h81-disko> booted_machine # [   10.004120] zfs-import-zstorage-start[642]: importing ZFS pool "zstorage"...Successfully imported zstorage
vm-test-run-disko-h81-disko> booted_machine # [   10.016772] systemd[1]: Finished Import ZFS pool "zstorage".
vm-test-run-disko-h81-disko> booted_machine # [   10.020708] systemd[1]: Reached target ZFS pool import target.
vm-test-run-disko-h81-disko> booted_machine # [   10.025414] systemd[1]: Mounting /zstorage...
vm-test-run-disko-h81-disko> booted_machine # [   10.028882] systemd[1]: Starting Mount ZFS filesystems...
vm-test-run-disko-h81-disko> booted_machine # [   10.102745] systemd[1]: Mounted /zstorage.
vm-test-run-disko-h81-disko> booted_machine # [   10.105224] systemd[1]: Mounting /zstorage/data...
vm-test-run-disko-h81-disko> booted_machine # [   10.108736] systemd[1]: Mounting /zstorage/share...
vm-test-run-disko-h81-disko> booted_machine # [   10.141732] systemd[1]: Finished Firewall.
vm-test-run-disko-h81-disko> booted_machine # [   10.171172] mount[900]: zfs_mount_at() failed: mountpoint or dataset is busy
vm-test-run-disko-h81-disko> booted_machine # [   10.173195] systemd[1]: zstorage-data.mount: Mount process exited, code=exited, status=2/INVALIDARGUMENT
vm-test-run-disko-h81-disko> booted_machine # [   10.177304] systemd[1]: zstorage-data.mount: Failed with result 'exit-code'.
vm-test-run-disko-h81-disko> booted_machine # [   10.185436] systemd[1]: Failed to mount /zstorage/data.
vm-test-run-disko-h81-disko> booted_machine # [   10.195107] systemd[1]: Dependency failed for Local File Systems.
vm-test-run-disko-h81-disko> booted_machine # [   10.208378] systemd[1]: local-fs.target: Job local-fs.target/start failed with result 'dependency'.
vm-test-run-disko-h81-disko> booted_machine # [   10.216897] mount[899]: zfs_mount_at() failed: mountpoint or dataset is busy
vm-test-run-disko-h81-disko> booted_machine # [   10.222705] systemd[1]: local-fs.target: Triggering OnFailure= dependencies.
vm-test-run-disko-h81-disko> booted_machine # [   10.225423] systemd[1]: systemd-ask-password-console.path: Deactivated successfully.
vm-test-run-disko-h81-disko> booted_machine # [   10.228617] systemd[1]: Stopped Dispatch Password Requests to Console Directory Watch.
vm-test-run-disko-h81-disko> booted_machine # [   10.232548] systemd[1]: systemd-ask-password-wall.path: Deactivated successfully.
vm-test-run-disko-h81-disko> booted_machine # [   10.236807] systemd[1]: Stopped Forward Password Requests to Wall Directory Watch.
vm-test-run-disko-h81-disko> booted_machine # [   10.239444] systemd[1]: Reached target Timer Units.
vm-test-run-disko-h81-disko> booted_machine # [   10.240638] systemd[1]: Reached target Login Prompts.
vm-test-run-disko-h81-disko> booted_machine # [   10.242816] systemd[1]: Reached target Preparation for Network.
vm-test-run-disko-h81-disko> booted_machine # [   10.244750] systemd[1]: Reached target All Network Interfaces (deprecated).
vm-test-run-disko-h81-disko> booted_machine # [   10.246527] systemd[1]: Starting Networking Setup...
vm-test-run-disko-h81-disko> booted_machine # [   10.248710] systemd[1]: Reached target Socket Units.
vm-test-run-disko-h81-disko> booted_machine # [   10.250470] systemd[1]: Reached target ZFS startup target.
vm-test-run-disko-h81-disko> booted_machine # [   10.251846] systemd[1]: Started Emergency Shell.
vm-test-run-disko-h81-disko> booted_machine # [   10.257317] systemd[1]: Reached target Emergency Mode.
vm-test-run-disko-h81-disko> booted_machine # [   10.258771] systemd[1]: Starting Create Volatile Files and Directories...
vm-test-run-disko-h81-disko> booted_machine # [   10.260449] systemd[1]: zstorage-share.mount: Mount process exited, code=exited, status=2/INVALIDARGUMENT
vm-test-run-disko-h81-disko> booted_machine # [   10.267525] systemd[1]: zstorage-share.mount: Failed with result 'exit-code'.
vm-test-run-disko-h81-disko> booted_machine # [   10.269648] systemd[1]: Failed to mount /zstorage/share.
vm-test-run-disko-h81-disko> booted_machine # [   10.271695] systemd[1]: Finished Mount ZFS filesystems.
vm-test-run-disko-h81-disko> booted_machine # [   10.319418] systemd[1]: Finished Create Volatile Files and Directories.
vm-test-run-disko-h81-disko> booted_machine # [   10.325848] systemd[1]: Starting Rebuild Journal Catalog...
vm-test-run-disko-h81-disko> booted_machine # [   10.341778] systemd[1]: Starting Userspace Out-Of-Memory (OOM) Killer...
vm-test-run-disko-h81-disko> booted_machine # [   10.353247] systemd[1]: Starting Network Time Synchronization...
vm-test-run-disko-h81-disko> booted_machine # [   10.363540] systemd[1]: Starting Record System Boot/Shutdown in UTMP...
vm-test-run-disko-h81-disko> booted_machine # [   10.400819] systemd[1]: Finished Record System Boot/Shutdown in UTMP.
vm-test-run-disko-h81-disko> booted_machine # [   10.491602] systemd[1]: Finished Rebuild Journal Catalog.
vm-test-run-disko-h81-disko> booted_machine # [   10.494829] systemd[1]: Starting Update is Completed...
vm-test-run-disko-h81-disko> booted_machine # [   10.541612] systemd[1]: Finished Update is Completed.
vm-test-run-disko-h81-disko> booted_machine # [   10.582904] systemd[1]: Finished Networking Setup.
vm-test-run-disko-h81-disko> booted_machine # [   10.583988] systemd[1]: Reached target Network.
vm-test-run-disko-h81-disko> booted_machine # [   10.585202] systemd[1]: Reached target Network is Online.
vm-test-run-disko-h81-disko> booted_machine # [   10.612229] systemd-oomd[926]: No swap; memory pressure usage will be degraded
vm-test-run-disko-h81-disko> booted_machine # [   10.617472] systemd[1]: Started Userspace Out-Of-Memory (OOM) Killer.
vm-test-run-disko-h81-disko> booted_machine # [   10.700820] systemd[1]: Started Network Time Synchronization.
vm-test-run-disko-h81-disko> booted_machine # [   10.702712] systemd[1]: Startup finished in 3.298s (kernel) + 7.401s (userspace) = 10.700s.
vm-test-run-disko-h81-disko> booted_machine # [   10.708247] (plymouth)[910]: emergency.service: Executable /nix/store/dzp7d4k1d94s1x49p9171mvcsfyxr7bj-systemd-254.6/bin/plymouth missing, skipping: No such file or directory
vm-test-run-disko-h81-disko> booted_machine # You are in emergency mode. After logging in, type "journalctl -xb" to view
vm-test-run-disko-h81-disko> booted_machine # system logs, "systemctl reboot" to reboot, or "exit"
vm-test-run-disko-h81-disko> booted_machine # to continue bootup.
vm-test-run-disko-h81-disko> booted_machine # Press Enter for maintenance

It seems that the filesystem is being mounted twice, however, I haven't defined any fileSystems options in the NixOS configuration. If I add mountpoint="legacy" to all datasets, the issue goes away. Full configuration: https://github.com/Toomoch/nixos-config/blob/4cbacfebd7a2cc31af913fb51fc2a45d93e54cf0/system/machine/h81/disko.nix

EDIT: apparently systemd and zfs are trying to mount it, is there a way to stop systemd from mounting it using disko?

nbdd0121 commented 6 months ago

I had the same issue with my home directory, and I ended up using fileSystems."/home".options = ["noauto"].

I tried remove mountpoint (but keep options.mountpoint) but it doesn't work very well because then it is not mounted by disko and so activation script can't access my home directory anymore.

An ideal solution is probably to change allow omitting mountpoint when non-legacy mountpoint is used and still have disko mounting it. Then if someone would like to omit this from fileSystems config, they can just not have mountpoint option.

iFreilicht commented 2 months ago

For anyone else hitting this issue and being confused about the solution:

When starting from the example above, you can change 

         data = {
            type = "zfs_fs";
            mountpoint = "/zstorage/data";
          };

to

         data = {
            type = "zfs_fs";
            options.mountpoint = "/zstorage/data";
          };

this means disko will not add the options that will mount the datasets via systemd, but the zfs option mountpoint will still be defined, and so if the zpool is imported, the dataset is automatically mounted. You might still have to add the pool manually by setting boot.zfs.extraPools = [ "zstorage" ]; (or whatever your pool name is).

The disadvantage of this is that it won't work if you need the dataset to be mounted immediately during the installation (like for /home), in which case the noauto flag as mentioned by @nbdd0121 would be the better solution.

However, if you're using datasets purely for data storage of other services, this won't be an issue, as systemd will make sure all mounts are completed before trying to start any userspace services.

nixos-discourse commented 2 months ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/how-do-i-actually-run-a-successful-installation-with-nixos-anywhere/49456/4