Open abathur opened 4 years ago
Update 1: I got these working, but pre-build-hook
only runs when sandbox = true
which is probably a non-starter for a general fix (whether adopted by default or recommended).
Update 2: I ran 30 timed runs of a short 7-build invocation using pre-/post-build hooks to disable assesments which averaged 47.9229s/run. Non-hook comparisons: 50.037s for builds with assessments on, 44.196s with them off. This seems to suggest ~266ms total overhead per hook call.
No sooner than I got this posted, @cole-h asked on IRC if pre-build-hook
and post-build-hook
could handle this. I wasn't aware of these. I assume the answer is yes, and I'll give it a try soon.
It looks like enabling/disabling assessments takes about 15-30ms per invocation on my system, so my interest in just doing it once for the invocation is in avoiding that overhead.
This issue has been mentioned on NixOS Discourse. There might be relevant details there:
https://discourse.nixos.org/t/macos-catalina-performance-issue/7359/12
Sigh. This is just an update.
I collected enough samples to be fairly sure this workaround doesn't significantly improve performance. (This is the only prospective workaround I'm currently aware of that Nix could implement--there's another that does work, but AFAIK there's no way to enable it without GUI access to the Security & Privacy preferences panel. It has also been causing some reboot problems for me on 10.15.4-5, though it might be fixed in 10.15.6).
I took a little time this weekend to tease out why...
spctl --master-disable
) is still doing some part of (but not all) of the assessment, but ignoring the result and reporting a pass. It is nowhere near as effective as adding the user's Terminal app to the Developer Tools exemption in Security & Privacy.spctl --master-disable
method only cut the runtime down to 27.5 minutesSince I already put together a list of projects that have that have Nix expressions building on macOS in CI, I'll pick a few of them out and run enough tests this week to brute-force collect the data that way...
This has been a PITA to collect data on.
My goal was to take the projects I previously tracked CI runs for and run 30 timed builds with an untimed garbage collection between each.
I had to weed out a few of the projects I used for collecting samples in CI because they were system/dotfile repos that were harder to test local. I also observed repeated hangs and cras/reboots in a few of the remaining builds. I haven't had time to babysit/debug builds lately, so I had to cut a few more out of the results for this, leaving me with good numbers on only 5 of the 12 projects I tracked on CI.
Here's the impact of using the developer tools exemption to disable assessments on the median runtime of 30 builds (on a 2018 i5 MBA) of these 5 projects:
Other notes:
I'll follow this comment with a bit of a thread summary.
I feel like this has mostly run its course for now; assuming this may go fallow until Apple changes something with these systems again, I'll summarize where this stands so that it's easier to catch up later :)
spctl --master-disable
, and adding the user's "terminal" app to the Developer Tools exemption in the Privacy panel. (White lie: it sounds a third option is using profiles to do this, but I gather it requires an MDM?) Unfortunately, I've come to the conclusion that neither of these (at least, in their current form) are worth recommending.
spctl --master-disable
only disables some fraction of the assessment process and has very little relative impact on the overhead they add. (Edit: confirmed this in latest Big Sur beta, too)@arianvp I've exhausted the options I'm aware to no avail. Curious if you think (just from the summary above) it's worth trying to pursue this with the dev-rel contact? I waffle between having really low expectations for this accomplishing anything, and thinking there are a lot of small things they could do that could all independently have a big impact here.
¹ A big part of the performance penalty is network overhead and I have no sense of how well geo-distributed the server(s) fielding these requests are. Some reports in the initial HN thread about this suggested fairly long international response times, so it may be worth noting that I'm in Houston and the benefit of circumventing these checks might vary by location, network, etc..⏎
I marked this as stale due to inactivity. → More info
I closed this issue due to inactivity. → More info
News of a macOS Catalina performance issue broke on HN a few weeks ago (https://news.ycombinator.com/item?id=23273247, https://news.ycombinator.com/item?id=23281564), and I've spent a little time trying to get a sense of how it impacts the performance of Nix on macOS and what workarounds may exist.
Issue itself
Potential Fix
The only "promising" workaround I've found (i.e., works in manual testing, doesn't have terrible UX, and might be something Nix can transparently implement) is if nix-daemon can disable the syspolicyd assessments (
spctl --master-disable
), run the entire set of builds that are required to satisfy the user's invocation of nix, and then re-enable them (spctl --master-enable
).Getting a better sense of the performance impact
It'll probably be easier to know if this is worth acting on with more data points.
I took a little time this weekend to find the active projects running macOS+Nix builds on GitHub Actions and write a script that can download and compile the stage run durations across different jobs.
I'm planning to send PRs to a few of these projects to see if they'll take on a temporary comparison job to generate a broader set of data points. If you have a CI job that fits the bill and can spare a few minutes to set it up, I've collected my settings and analysis script in a gist.