Feedback for NixOS test

[Luflosi]

The last missing piece for perfect NixOS integration is a NixOS module. This will allow awesome things like specifying

fileSystems."/mnt/apfs" = {
  device = "/dev/disk/by-partlabel/apfs";
  fsType = "apfs";
}

in the configuration, which is all that is required to automatically mount the filesystem during boot, no need to install anything or configure anything else. As part of writing a NixOS module, it is good practice to write a test for a basic sanity check and to gain some confidence, that at least the basic stuff all works as expected. Here is how the test looks currently:

import ./make-test-python.nix ({ pkgs, ... }: {
  name = "apfs";
  meta.maintainers = with pkgs.lib.maintainers; [ Luflosi ];

  machine = { pkgs, ... }: {
    virtualisation.emptyDiskImages = [ 1024 ];

    boot.supportedFilesystems = [ "apfs" ];
  };

  testScript = ''
    machine.wait_for_unit("basic.target")
    machine.succeed("mkdir /tmp/mnt")

    with subtest("mkapfs refuses to work with a label that is too long"):
      machine.fail( "mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7F' /dev/vdb")

    with subtest("mkapfs works with the maximum label length"):
      machine.succeed("mkapfs -L '000102030405060708090A0B0C0D0E0F101112131415161718191A1B1C1D1E1F202122232425262728292A2B2C2D2E2F303132333435363738393A3B3C3D3E3F404142434445464748494A4B4C4D4E4F505152535455565758595A5B5C5D5E5F606162636465666768696A6B6C6D6E6F707172737475767778797A7B7C7D7E7' /dev/vdb")

    with subtest("Enable case sensitivity and normalization sensitivity"):
      machine.succeed(
          "mkapfs -s -z /dev/vdb",
          # Triggers a bug, see https://github.com/linux-apfs/linux-apfs-rw/issues/15
          # "mount -o cknodes,readwrite /dev/vdb /tmp/mnt",
          "mount -o readwrite /dev/vdb /tmp/mnt",
          "echo 'Hello World 1' > /tmp/mnt/test.txt",
          "[ ! -f /tmp/mnt/TeSt.TxT ] || false", # Test case sensitivity
          "echo 'Hello World 1' | diff - /tmp/mnt/test.txt",
          "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt",
          "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt",
          "[ ! -f /tmp/mnt/\u00e1.txt ] || false", # Test Unicode normalization sensitivity
          "umount /tmp/mnt",
          "apfsck /dev/vdb",
      )
    with subtest("Disable case sensitivity and normalization sensitivity"):
      machine.succeed(
          "mkapfs /dev/vdb",
          "mount -o readwrite /dev/vdb /tmp/mnt",
          "echo 'Hello World Hello World Hello World' > /tmp/mnt/Test.txt",
          "echo -n 'Hello World' > /tmp/mnt/test.txt",
          "echo ' 1' >> /tmp/mnt/TEST.TXT",
          "umount /tmp/mnt",
          "apfsck /dev/vdb",
          "mount -o readwrite /dev/vdb /tmp/mnt",
          "echo 'Hello World 1' | diff - /tmp/mnt/TeSt.TxT", # Test case insensitivity
          "echo 'Hello World 2' > /tmp/mnt/\u0061\u0301.txt",
          "echo 'Hello World 2' | diff - /tmp/mnt/\u0061\u0301.txt",
          "echo 'Hello World 2' | diff - /tmp/mnt/\u00e1.txt", # Test Unicode normalization
          "umount /tmp/mnt",
          "apfsck /dev/vdb",
      )
  '';
})

Most of the space in the file is taken up by a Python script. The strings passed to machine.succeed() are shell script one-liners that all need to succeed for the test to succeed. The test builds a qemu VM with an empty 1GB disk image, loads the APFS kernel module and then executes the tests. Here is the NixOS test for ZFS. It basically tests smb file sharing, encryption and forefully importing a pool on a machine with a different host ID, for two versions of ZFS and the Linux kernel. I'd like to hear your feedback, if you think this is an adequate amount of testing or if you would test more or test less? Should I maybe reboot the VM somewhere during the test? Maybe test long filenames? Call apfsck more often? Create a whole bunch of files and see if any disappear? Something else entirely?

[eafer]

Cool. I think your tests are enough if they are just intended for the package. Maybe you could unmount and remount the device before checking the file contents, to confirm that they are persisted. But there's probably no point in going further, there are too many edge cases and we have xfstests for dealing with those.

[Luflosi]

Thanks for your feedback and time. I'm already unmounting and remounting the device once before checking the file contents. I don't think doing that every time before checking the file contents provides any additional value. In case you're interested, here is my pull request to Nixpkgs: https://github.com/NixOS/nixpkgs/pull/141122.