search

LnL7 / nix-darwin

nix modules for darwin
MIT License
2.82k stars 431 forks source link

Add `nix.optimise` module #915

Closed malob closed 6 months ago

malob commented 6 months ago

Add nix.optimise based of NixOS module. Implementation in nix-darwin based of nix.gc.

Would appreciate a sanity check on changes to module/system/checks.nix in particular since I'm least certain about that.

malob commented 6 months ago

Supersedes #588

Samasaur1 commented 6 months ago

The service should probably use /bin/wait4path

malob commented 6 months ago

@Samasaur1, thanks! I can do that.

As an aside, should nix.gc use it as well?

Samasaur1 commented 6 months ago

Yeah, it probably should, but this PR should remain focused on adding nix.optimise. There's an open issue (https://github.com/LnL7/nix-darwin/issues/774) about making this change treewide

malob commented 6 months ago

Alrighty, I think this is ready to go.

emilazy commented 3 months ago

As much as I appreciate the work to improve NixOS compatibility here, I think this module is a huge footgun and that we should remove it, or at least add very loud scary warnings when it’s turned on. Store optimization is basically completely broken on macOS and causes many bizarre and hard‐to‐debug issues; cf. https://github.com/NixOS/nix/issues/7273 https://github.com/DeterminateSystems/nix-installer/pull/542 https://github.com/LnL7/nix-darwin/issues/693. I have run into explicit breakage every time I have tried to use it, but what’s worse is that it can fail in ways that could cause silent corruption (as far as I can tell it’s only luck that made that last issue result in crashes rather than buggy behaviour). It's at the top of my list when someone has strange behaviour that seems like it violates the Nix model, so I don’t think that anyone on macOS should have this option turned on in its current upstream state.

emilazy commented 3 months ago

(Just to add: I know some of the issues are more prominent with auto-optimise-store vs. a batch process, presumably due to a lower chance of race conditions, but I’ve run into serious problems along these lines even with manual nix-store --optimise runs and have consequently given up on the feature entirely on macOS.)

malob commented 3 months ago

Interesting. I implemented this because of the issues with auto-optimise-store, and haven't myself experienced, or heard of, issues with manually running store optimization. How prominent is the latter? Are these issues documented anywhere online?

emilazy commented 3 months ago

I’ll give an immediate mea culpa here: I have much clearer memories and confidence in automatic optimization being broken on macOS than manual, and it’s entirely possible that I’m misremembering running into unambiguous issues with manual optimisation. So, sorry for jumping the gun on this one – I have an instinctive twitch reaction to the topic of store optimization at this point, and it got ahead of me in this case.

To be more explicit about my levels of confidence: I know for certain that automatic store optimization on macOS is horribly broken and can lead to scary corruption, and that even manual store optimization can break things on some platforms (e.g., Linux with SELinux). I suspect that manual optimization on macOS can probably cause similar issues to automatic store optimization if it happens to race with a build. I vaguely recall that I did manual store optimization on macOS and then gave up due to issues, but it’s possible I’m confusing it with the SELinux case. Unfortunately I don’t have great documentation about all these issues (even “automatic store optimization is broken on macOS” is basically folklore, sadly – there should probably be upstream documentation about it, or even a warning).

Seems like the best way forward here is to see if this causes any issues when people use it; I might consider turning it on myself to be a guinea pig. The disk savings are nice; I had just resigned myself to missing out on them because of all the issues I’ve had in the past.

malob commented 3 months ago

No worries, I’ve been driven half crazy in the past trying to debug issues on macOS arising from enabling auto-optimise-store, so I definitely get your concern 🙂.

FYI, as far as I’m aware, the best “documentation” of issues with that on macOS is this issue: https://github.com/NixOS/nix/issues/7273

Your plan here sounds reasonable to me. I’ve got nix.optimise enabled, so I’ll make sure to keep an eye out for any issues myself.

emilazy commented 3 months ago

Yeah, I’m mostly just twitchy because it’s such a nondeterministic thing, and I worry that making it a scheduled batch process only works more reliably because it reduces the likelihood of it racing with a build. If only we could use something like Antithesis to try and get a deterministic reproducer…