Flake copying performance regressed on macOS in 2.19

NixOS / nix

Nix, the purely functional package manager

https://nixos.org/

GNU Lesser General Public License v2.1

11.94k stars 1.46k forks source link

Flake copying performance regressed on macOS in 2.19 #11098

Open cigrainger opened 1 month ago

cigrainger commented 1 month ago

Describe the bug

I'm on nix-darwin and copying to the store can take several minutes. This first appears on 2.19. 2.18 is fast (a few seconds). E.g. nix run nixpkgs#scc takes >5 minutes.

Steps To Reproduce

Set nix.package to any version > 2.18 and try nix run nixpkgs#scc or nix flake update.

Expected behavior

The speed should be consistent after 2.18.

nix-env --version output

nix-env (Nix) 2.19.6

Additional context

I'm on nix-darwin.

Priorities

Add :+1: to issues you find important.

emilazy commented 1 month ago

It isn‘t 5 minutes bad for me, but I definitely also noticed a regression in flake copying performance on my Intel MacBook Pro after switching from 2.18 to 2.23 to pick up the Darwin sandbox fix. That‘s already one of the most painful parts of flakes, especially when working on Nixpkgs, so it‘s very unfortunate.

Montmorency commented 1 month ago

Also noticing this regression in flake performance and /nix/store interactions. [Current config is here] (https://github.com/Montmorency/nix-darwin).

nixos-discourse commented 1 month ago

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-07-17-nix-team-meeting-minutes-162/49255/1

Mic92 commented 1 week ago

I cannot reproduce this:

$ nix --version   
nix (Nix) 2.23.3
$ time nix eval --store $HOME/test-store nixpkgs#nix.version
nix eval --store $HOME/test-store nixpkgs#nix.version  1.86s user 7.07s system 50% cpu 17.615 total

[nix-shell:~/git/nixpkgs]$ sudo rm -rf $HOME/test-store/   
[nix-shell:~/git/nixpkgs]$ nix --version
nix (Nix) 2.18.5                       
[nix-shell:~/git/nixpkgs]$ time nix eval --store $HOME/test-store nixpkgs#nix.version        
"2.18.5"
real    0m22.181s
user    0m2.058s
sys     0m17.071s

In this case the old version was even slower, but my system is also more under load running some nix-builds at the same time.

roberth commented 1 week ago

We might be holding libgit2 wrong. I'm looking into packfile generation to reduce the vast amount of small I/O.

worldofgeese commented 3 days ago

Just another anecdote: with Nix 2.33.3 I'm seeing hours-long copy operations. I'd like to provide more substantive logs but the most I get is copying '/Users/$USER/.local/share/flake-config/' to the store.

roberth commented 1 day ago

2.33.3

Either geese can travel through time, or this has a typo. Which version did you run? Did it complete, or did you interrupt Nix?

If you could get stack traces with a debugger, that may help narrow down the cause.