A Nix store that works well on Windows

[roberth]

Is your feature request related to a problem? Please describe.

I don't know how we'd best present a store on windows. I'm anything but an expert on this, so I can only hope to provide seeds for a productive discussion.

Possible solutions

• C:\nix\store: works on most systems?

  • Requires admin permissions to modify C:\
  • Somewhat dependent on physical location
    • NTFS symlinks can do something, but are not opaque to programs
      • => more impurity

• \\nix\store

  • How's support for network drives in the windows ecosystem? Not great?

    • When using CMD, cd \wsl$\Ubuntu\home will not work (as CMD does not support UNC paths as current directories), however copy \wsl$\Ubuntu\home\somefile.txt C:\dev\ will work -- Microsoft dev blog

    • Nix store as a forcing function to improve the windows ecosystem to adopt UNC more? Nix is supposed to be ambitious?
  • Probably requires admin permissions too
  • Looks nice
    • More virtual than a drive letter
  • Independent of physical location
    • Presumably the API we use to achieve this allows us to present the store in a very low entropy manner - very opaque, which is great for reproducibility
  • Implementation?
    • SMB? Possibly slow]
    • WebDAV seems to be an alternative. This may be easier to work with, but not a substantial user facing improvement over SMB?
    • API that backs SMB and/or WebDAV? Does that exist? Haven't found it yet.
    • WSL seems to use 9P file protocol!
      • Microsoft dev blog announcement with some details

Priorities

Add :+1: to issues you find important.

[Ericson2314]

https://en.m.wikipedia.org/wiki/SUBST / https://learn.microsoft.com/en-us/windows/win32/api/fileapi/nf-fileapi-definedosdevicew are old things that allow us to mount directories as drives. This will keep MS-DOS paths working.

I do think we could end up using more UNC paths for path length reasons, however.

[Ericson2314]

Therefore, I think we will be best off with

N:\x\store

Note that it is the same length as

/nix/store

which can allow for some janky rewrites :)

[roberth]

: looking like i is clever.

Still not a fan of "just" claiming a drive. This could be a huge blocker in established environments.

Can a network drive symlink to an arbitrary path? If so, that might be a workaround in such situations, but the workaround will be observable. Granted, Windows software is less aware of the existence of symlinks, as those were not traditionally part of the operating system.

same length

Should the UNC path then be \\ix\store?

(\\ looks like n if you squint really hard...)

[Ericson2314]

Still not a fan of "just" claiming a drive. This could be a huge blocker in established environments.

Well, remember we're not claiming the whole drive, just the x subdirectory at the root of it. I hope that is OK, and for situations where it really is not, they hopefully have enough resources to rebuild the world with a different store path?

Can a network drive symlink to an arbitrary path? If so, that might be a workaround in such situations, but the workaround will be observable. Granted, Windows software is less aware of the existence of symlinks, as those were not traditionally part of the operating system.

That seems like something that should work too. And yeah I think both will be observable so it's an OK place to start. I just worry that supposedly lots of software does not work with UNC paths.

[ChristineBoersen]

You have to give the user the choice of drive in windows environment. The old cmd.exe doesn't like unc path for shell prompts, but powershell is fine with that. Windows users typically mount another drive letter for a UNC path to get around those limitations

Windows isn't as happy with symlinks and networks as linux is, hence allow any drive to be specified. :)

[Ericson2314]

@ChristineBoersen I think we would indeed allow a user the choice of drive and directory within that drive, but a non-standard choice will result in not being able to use the cache. (At least, safely use the cache. Sedding is possible but only semi-reliable as it can mess up orderings, checksum, etc.)

https://github.com/NixOS/nix/issues/9549 I wrote this up for the much harder "true" solution of store objects that simple aren't allowed to know their own store dir, so we can be sure they are safe to relocate.

[the-sun-will-rise-tomorrow]

Hi! I am eager to see Nix on Windows. I hope you'll find this context useful:

One category of use cases for Nix is when the user (human using the computer) does not interact with Nix directly; rather, it's just a dependency of some dependencies of software that they need in order to do their job. This can become a likely case in corporate environments, when the system administration / devops / etc. team sets up software for non-technical users.

In corporate environments, Windows drive letters have very strong semantic significance. "The J drive" or "The P drive" is common lingo. These are usually configured to be mapped to network shares by system administrators.

As such, I think any proposal to claim a drive letter for the project's use is going to have a non-trivial risk of significantly impacting Nix adoption in such environments, for these reasons:

• There is a real, non-trivial chance that any particular drive letter will already be claimed by the organization's existing setup, and long incorporated into users' training. It will also likely be a drive letter that can't host the Nix store, i.e. a mapped network share.
• Because drive letters are highly visible to users, the addition of a new drive will be a very visible change, one that users will need to be told to ignore.
• In situations where Nix is used as a dependency for software provisioning, the high visibility impact of the drive letter requirement can be seen as disproportionate to the perceived benefit added by Nix, especially during initial adoption.

I would especially not recommend using N:\x. It is non-descriptive and difficult to search for.

C:\nix\store and \\nix\store (if it can be made to work) sound great to me!

Requires admin permissions to modify C:\

I believe that creating new directories in C:\ is usually allowed for unprivileged users.

However, a hypothetical "Nix for Windows" will be distributed in a format that's common for the platform, i.e. a Windows installer, right? In which case, requiring administrator permissions to install it is expected. It would also allow configuring permissions for the Nix store so that only the Nix daemon can write to it.

Hope this helps!

[Rucadi]

Hi!

I just finished trying some nix evals in Windows with the amazing job done by @Ericson2314 .

I'm pretty hyped for this, but I have several concerns about the Nix Store.

• %systemdrive% is 99% of the times C:\ , but windows can be assigned to any letter and C:\ may not exist
• Low space in C:\ , a lot of laptops decide to have a low-capacity SSD and a higher capacity HDD (usually D:)
• Not always able to create a folder on C:\
• Multiuser Windows with different accounts may not want to share nix store
• Different FS, like btrfs, may want to be used for nix store but not for C:\
• I also use things like https://www.romexsoftware.com/en-us/primo-cache/ which I may only want to use for the nix store

Also, this concern seems valid:

Windows isn't as happy with symlinks and networks as linux is, hence allow any drive to be specified. :)

Maybe making the nix store a vhdx mounted into a C:\nix is a good solution, it would mean that it can be easily replaced by any fs, easily copied and easy removed, but that may be more an user choice than a nix-lang design choice.

Another alternative, although it may seem a bit hacky is to use https://github.com/ModOrganizer2/usvfs MO2 team faced the problem of "symlinks on windows", and the solution for them was to hook the API to present a virtual fs.

This approach could also be used for nix store on windows, which would mean that we don't care where the store is anymore. However, this introduces complexity and points of failure, I would only consider this if symlinks become a problem when interacting with apps in Windows.

The most obvious choice however seems to be C:\nix, but maybe we can also get away with %appdata% on windows?

Other concern that comes to mind is the usual lenght-limit for paths: https://learn.microsoft.com/en-us/windows/win32/fileio/maximum-file-path-limitation?tabs=registry

Which means that we have to tweak the windows registry to allow large paths, and minimum supported version may be Windows 10, version 1607