flakes: avoid copying local flake to the store when `self` arg is omitted

[lilyball]

Is your feature request related to a problem? Please describe. I wanted to put a flake.nix in the parent folder that contains a bunch of projects, including their build artifacts. I didn't have any need for a source tree, this was purely to power nix-direnv (it just had devShells, one per project).

The problem is attempting to evaluate the flake, even with nix develop, is it immediately tried to copy the whole source tree to the Nix store. This isn't something that was mentioned in the nix flake documentation. If this were a git repo then presumably it would only copy tracked files (though that can be bad enough), but in my case as this was just a path flake it tried to copy the hundreds of thousands of files comprising many gigabytes of data (as it included build artifacts).

Describe the solution you'd like At a minimum this behavior needs to be made clear. But I'd also like to be able to just omit the self arg to the outputs function. #3121 suggests copying to the store lazily, which would be excellent. But barring that, omitting the self arg guarantees I'm not using the source tree and so it should avoid copying it.

I'm not sure offhand how this should handle the flake wanting to import other files via local paths. Like #3121 this might require verifying the file against git ls-files for a git repo, though for a path

Describe alternatives you've considered Another alternative is to let me set inputs.self.sourceFilter to a (path: type: …) lambda that will be given to builtins.filterSource. This would be used to filter the source tree, with the exception that flake.nix will always be included (either that or an error if it's not).

The way this could work is that flake.nix is evaluated in its current location with restrictions such that it cannot import any other files, and the inputs.self.source attribute (if defined) would be evaluated to produce the source tree used to copy to the store. Then flake evaluation would proceed as normal from there.

[lilyball]

My original alternative thought was actually to have inputs.self.source = [ ./helper.nix ./utils ./foo/bar/nix ] but I wasn't sure how that would interact with git ?dir= flakes, as the source tree appears to still be the git repo root but it would be surprising if the above source specification resulted in a source tree of ./dir/helper.nix instead.

[TLATER]

The way this could work is that flake.nix is evaluated in its current location with restrictions such that it cannot import any other files, and the inputs.self.source attribute (if defined) would be evaluated to produce the source tree used to copy to the store. Then flake evaluation would proceed as normal from there.

If something like this is implemented, a helpful hint to point out that feature exists when you forget/don't know about it and try to import a file from the local directory would be very user friendly.

[haroldcarr]

Is there any more info on this copying and how to avoid it?

I'm concerned that if I develop via nix develop that I will end up creating many different hash directories in /nix/store with mostly the same stuff modulo changes.

[erikarvstedt]

@haroldcarr, fixing this is part of the nix-2.8 milestone. See https://github.com/NixOS/nix/issues/3121.

[j-baker]

@lilyball I'm not sure if you found a good workaround to this issue yet, but this felt pretty slick for the nix-direnv problem. https://github.com/direnv/direnv/issues/1103. Interested to hear if you have a nicer workaround, I've seen your various issues and proposals on this topic and they're in general very reasonable and seemingly relatively straightforward to implement.

[lilyball]

@j-baker The workaround I settled on was just moving my flake into a subdirectory of the parent folder.

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/minecraft-development-environment-both-fabric-forge-unable-to-find-libraries-at-runtime/36999/3