search

NixOS / nix

Nix, the purely functional package manager
https://nixos.org/
GNU Lesser General Public License v2.1
12.12k stars 1.47k forks source link

scopedImport memoization #1212

Open copumpkin opened 7 years ago

copumpkin commented 7 years ago

In this commit introducing scopedImport, @edolstra mentions that scopedImport doesn't memoize file loading. I imagine that scopedImport with memoization would need to maintain a map from (overrides, filename) to loaded Nix as opposed to just one from filename, but would otherwise be pretty similar. It does feel potentially icky if overrides has a function in it, as it's almost certain to have, because there's not a sensible way to compare those for equality other than by AST representation, which I think we'll have lost by that point. In some ways this issue is similar to the one we have with filterSource in the deterministic evaluation work (which is incidentally what caused me to look at this).

Either way, I think it would be nice to fix this, even with a super simplistic "compare functions by address in memory" heuristic right now. In practice I expect we won't be importing the same files with different environments all that often, and it feels like the most likely scenarios would be:

  1. scopedImport for small packages that don't need to recursively replace environments
  2. A recursive scopedImport that ensures that the overrides get plugged in across the entire evaluation by overriding import and scopedImport and builtins, etc.
copumpkin commented 7 years ago

@shlevy @edolstra do you know if the memoization thing is still an issue? I was thinking it would be pretty fun to "disable" angle-bracket usage inside nixpkgs with scopedImport, as another example.

shlevy commented 7 years ago

Yes, it's still an issue. Honestly I think "compare the whole overrides set by address" isn't the worst idea, given that we usally just do the overrides once at a top-level entry point (and we're not a moving GC)

stale[bot] commented 3 years ago

I marked this as stale due to inactivity. → More info

roberth commented 3 years ago

It's possible to perform memoization externally via let binding. If memoizing via let is not an option, it seems unlikely that the scope addresses will match.

I do see one case where that may be hard, which is where a scope item is a parameter (whose values do not make sense to enumerate in advance). That is a more general problem though, which can be solved by a function memoization primitive. That'd be more useful.

Don't make me use scopedImport as a means for function memoization ;)

stale[bot] commented 2 years ago

I marked this as stale due to inactivity. → More info

infinisil commented 1 year ago

There's now an issue relating to memoization of function application: https://github.com/NixOS/nix/issues/6228. I also opened https://github.com/NixOS/nix/issues/8024 to describing some problems related to scopedImport.

I think this issue can be closed.