The state of Nix on OS X

[dhess]

This is not meant as a rebuke or a criticism, it's just an honest question with no ill will: are there currently any Nix maintainers who are actively working on the OS X port?

I ask because it seems that, lately, many OS X-related issues go weeks or months with nothing more than a non-committal acknowledgement of the problem.

Examples of such long-standing issues, just off the top of my head:

• OS X sandbox builds
• https://github.com/NixOS/nixpkgs/issues/12863 (there is (or was, anyway) even a patch which fixes this one)
• https://github.com/NixOS/nixpkgs/issues/17372

In some cases, e.g., https://github.com/NixOS/nixpkgs/issues/18461 and #17372, these are show-stopper issues that affect, or will soon affect, nearly everyone running the OS X port. #17372 was reported over 6 weeks ago and as far as I can tell, no Nix maintainer even tried to reproduce the problem until a few days ago. Now we are just days away from the 10.12 release and we're only just starting to get a handle on things that are broken/will break.

Again, this is not a criticism, it's just a statement of fact. I understand that Nix is run by volunteers and that most of those volunteers are running NixOS or some other Linux distro, and therefore those platforms are the priority. If nobody has time to maintain Nix on OS X at the moment, I completely understand that and I am not looking to apportion blame; I would just appreciate a clearer picture of the current "state of the port."

[mpscholten]

Any ideas on how we could improve the situation?

For package related problems (like #12863) we could improve the situation with better testing. As travis is already running tests on macOS since a few months, package build failures should already decrease over time (because in theory travis will notify in case a PR breaks a package on macOS).

[copumpkin]

The state of the port is that it needs more people developing it, as opposed to just using it, but that I use Nix on OSX daily and tend to fix any outstanding issues that get in my way. Of course, my usage patterns are often different from other people's, and as you say I haven't had any time to install Sierra to see what was wrong with that bug.

Anyway, it's not great today, but most things I decide to try to install that aren't OSX-specific just seem to work. I encourage people to file tickets in the hopes that someone will pick them up, but they mostly just seem to accumulate. I'd welcome any help you can provide or suggestions on what the issue is.

[copumpkin]

TBC, I don't expect people to be tinkering with the stdenv at this point, but I'm slowly making progress (doing more work on it today finally after finding some time) on improving the stdenv to attempt to make it easier for other people to develop. But all the individual package issues seem like other people could handle them.

[dhess]

It's apparent that the port needs more active developers, so ultimately the only way to improve the situation is to get them.

How does the project get more active developers? The obvious way would be to make it easier, and more encouraging, for users to ramp their way up to becoming contributors. Suggestions on how to do that:

a) Better documentation on how the OS X stdenv works. As @copumpkin alludes, trying to fix stdenv issues comes with a very steep learning curve. I've tried on several occasions and end up getting lost in complexity and multiple layers of tools and variables, and there isn't much documentation out there to assist a prospective stdenv hacker. For a concrete example of this problem, take a look at the comments for #18461 where @copumpkin and @kirelagin are trying to fix the libiconv issue; the knowledge of why libiconv is being re-exported has been lost/is not easily found, even by core developers.

b) Get support for OS X sandboxed builds over the finish line and actually working. Wasn't the whole point of the pure OS X stdenv to isolate us from Apple issues such as the ones we're currently experiencing in #18461 and #17372? A more robust stdenv would mean that stdenv issues came up less often. Sandboxed builds got so close to being ready, and if I recall correctly were even recommended at one point, but since then, it seems that practically nothing has happened on that front. I tried to enable them last night in the hopes of working around #18461 and couldn't even get the first dependency to build.

c) Be more responsive to bug reports and patches. Example: I mentioned #12863 earlier because it's one that I personally reported and submitted a patch for. That was on Feb 7 and the issue is still not resolved. The patch I submitted literally adds a single linker flag to the build, a flag that is used on many other OS X packages in Nix, as well as being known to resolve the issue in upstream emacs, so it should not be controversial; yet here we are 7 months later and the issue is still open. I don't know how often this happens, but if you're asking users to help by becoming developers, examples like this one are not encouraging them to do so.

Speaking from personal experience, things like b) and c) lead to a bit of "broken window" syndrome where users of Nix on OS X (and also prospective users who are checking out the project to decide whether they want to start using it) get the impression that the project is not reliable enough for production use on their platform, which is a shame, because when it works, it it easily the best package management system for OS X. (edit: and the reason I mention this is that it, too, does not do much to encourage new users, who could conceivably become developers over time.)

[copumpkin]

Your points mostly make sense to me, but the real question for me is "who should be doing what differently". A lot of what you're saying makes sense in the abstract but I don't know who should be doing those things. We did have sandboxed builds working well for a long time (it was all I used for a while, and I did the initial work to make them work) but they broke and I have far less time now than I did when I first put this whole thing together. In the absence of me doing all of the above, who makes it happen and how? Documentation certainly helps but if I had eager Darwin developers breathing down my neck aching to help fix the stdenv I might allocate more of my free time to doing that instead of fixing the things myself. Since I don't, it becomes more of a "I'll do it once I get it simplified so I don't have to document as much" type deal rather than my #1 priority. If you or anyone else wants to step up and be the Darwin "taskmaster" in the same way @domenkozar is for NixOS (knows the system deeply and can keep the various threads organized), I think that would be a good start, but these sorts of "abstract nice-to-haves" are just frustrating to me when I see so few other people stepping up to help with them. On Sun, Sep 11, 2016 at 15:09 Drew Hess notifications@github.com wrote:

It's apparent that the port needs more active developers, so ultimately the only way to improve the situation is to get them.

How does the project get more active developers? The obvious way would be to make it easier, and more encouraging, for users to ramp their way up to becoming contributors. Suggestions on how to do that:

a) Better documentation on how the OS X stdenv works. As @copumpkin https://github.com/copumpkin alludes, trying to fix stdenv issues comes with a very steep learning curve. I've tried on several occasions and end up getting lost in complexity and multiple layers of tools and variables, and there isn't much documentation out there to assist a prospective stdenv hacker. For a concrete example of this problem, take a look at the comments for #18461 https://github.com/NixOS/nixpkgs/issues/18461 where @copumpkin https://github.com/copumpkin and @kirelagin https://github.com/kirelagin are trying to fix the libiconv issue; the knowledge of why libiconv is being re-exported has been lost/is not easily found, even by core developers.

b) Get support for OS X sandboxed builds over the finish line and actually working. Wasn't the whole point of the pure OS X stdenv to isolate us from Apple issues such as the ones we're currently experiencing in #18461 https://github.com/NixOS/nixpkgs/issues/18461 and #17372 https://github.com/NixOS/nixpkgs/issues/17372? A more robust stdenv would mean that stdenv issues came up less often. Sandboxed builds got so close to being ready, and if I recall correctly were even recommended at one point, but since then, it seems that practically nothing has happened on that front. I tried to enable them last night in the hopes of working around #18461 https://github.com/NixOS/nixpkgs/issues/18461 and couldn't even get the first dependency to build.

c) Be more responsive to bug reports and patches. Example: I mentioned

12863 https://github.com/NixOS/nixpkgs/issues/12863 earlier because

it's one that I personally reported and submitted a patch for. That was on Feb 7 and the issue is still not resolved. The patch I submitted literally adds a single linker flag to the build, a flag that is used on many other OS X packages in Nix, as well as being known to resolve the issue in upstream emacs, so it should not be controversial; yet here we are 7 months later and the issue is still open. I don't know how often this happens, but if you're asking users to help by becoming developers, examples like this one are not encouraging them to do so.

Speaking from personal experience, things like b) and c) lead to a bit of "broken window" syndrome where users of Nix on OS X (and also prospective users who are checking out the project to decide whether they want to start using it) get the impression that the project is not reliable enough for production use on their platform, which is a shame, because when it works, it it easily the best package management system for OS X.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NixOS/nixpkgs/issues/18506#issuecomment-246197993, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAKP_PeWc7y20wBleP01WZC7fDa8GtLks5qpFHwgaJpZM4J57mR .

[dhess]

@copumpkin You ask, "Who should be doing what differently?" Who are you asking? Me? Other Nix project members? Is it rhetorical?

[copumpkin]

More rhetorical, yeah. I honestly don't know and would love to hear ideas on how to improve things and draw more people in. Your list is a start but I don't know how to make it into reality without taking a few weeks off from my "real life" and dedicating them to Nix. It's hard for me to justify investing in that when even the Darwin "low-hanging fruit" (i.e., things that don't rely on specialized darwin stdenv knowledge) don't often much volunteer help. I'd love help and ideas but I think a strict list of todos only scratches the surface of the issues at play here. On Sun, Sep 11, 2016 at 15:42 Drew Hess notifications@github.com wrote:

@copumpkin https://github.com/copumpkin You ask, "Who should be doing what differently?" Who are you asking? Me? Other Nix project members? Is it rhetorical?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/NixOS/nixpkgs/issues/18506#issuecomment-246199937, or mute the thread https://github.com/notifications/unsubscribe-auth/AAAKP3eaBDmj-UyTIBUYcbgIQE9QBZvkks5qpFmggaJpZM4J57mR .

[copumpkin]

Anyway sorry, I probably sound more doom-and-gloom than I intended, and I do appreciate you caring about the state of the ecosystem. There are several major problems, not least that I think in principle a pure darwin stdenv is a much gnarlier problem than a pure Linux one, but we have far fewer developers and a very active linux community that regularly breaks our stuff because they (legitimately) don't really have a way to test it (except by waiting for Hydra). These all have solutions but they all require time and people and that's the most scarce resource here 😄

But anyway, as we speak I'm trying to upgrade and fix/simplify the stdenv, which is an iterative process but should (over time) simplify the thing enough to be more manageable/pick-uppable. But consider that even on the Linux side, the number of people who know what they're doing in the stdenv is pretty minimal, so we'll still be needing people to step up even after it gets simpler.

[dhess]

@copumpkin Again, let me reiterate in all sincerity that I am not criticizing you or any other project member. I personally have run an open source project with a reasonable number of users, and I know how frustrating it can be, and how much pressure it can bring down on your shoulders.

My reason for opening this issue was to get better visibility on the state of the OS X port, not to attempt to solve any of the problems currently dogging the port, per se. My list of suggestions was not intended to be prescriptive, nor to be comprehensive. I only provided it in an attempt to be helpful and responsive when asked, "What do you suggest?"

I think this thread has already been useful as it has made apparent the fact that you want/need help with the port. Re: why you don't get more volunteer help, I suspect it's a bit of a chicken-and-egg problem: it's difficult as a user to fix "big" OS X issues because of the complex and mostly undocumented stdenv, which in turn means you're not incentivized to spend time fixing the big issues (esp. stdenv) because you don't get much help.

So my answer to your mostly-rhetorical question re: "who should be doing what differently?" is that you can probably do the most good by spending most of the time you have for OS X on Nix in documenting or improving the current stdenv to the point that more people can help you out in the longer term.

[dhess]

Does Nix have an official organization, a la Debian, which oversees the project, accepts donations, etc?

[copumpkin]

This is the closest you'll find to that: https://nixos.org/nixos/foundation.html.

[dhess]

Yes, that is exactly the kind of thing I was looking for, thanks.

[copumpkin]

Again, let me reiterate in all sincerity that I am not criticizing you or any other project member. I personally have run an open source project with a reasonable number of users, and I know how frustrating it can be, and how much pressure it can bring down on your shoulders.

Thanks, and I didn't mean to come across as defensive. This has just been a pain point for a while and I don't really know what to do about it.

I'm going to proceed with the upgrade/simplify/document plan on the current stdenv. That means that docs won't be the first thing I produce, but won't be too far off either. I do want to get us off the 10.9-based stdenv that we made a couple of years ago, and get rid of many of the hacks we put in place back then in the process. I think the end result will have less to document and understand.

[dhess]

That sounds like a good plan to me, and is exactly how I would proceed with things if I were in your position.

[vcunat]

Does Nix have an official organization, a la Debian, which oversees the project, accepts donations, etc?

Well, it's purpose is still rather restricted, as described on that link.

I don't think most contributors need to really understand how stdenv works, as long as it works (kinda) and they know how to use it.

[copumpkin]

I don't think most contributors need to really understand how stdenv works, as long as it works (kinda) and they know how to use it.

In general agree, and I think most Linux contributors don't understand the pure linux stdenv either. Unfortunately some of the oddness in the darwin stdenv occasionally leaks into user packages and confuses people, but I do think that's less common, and it'll go down as I simplify and flesh out the stdenv bootstrap process.

[nixy]

@copumpkin If people were interested in understanding/contributing to the darwin stdenv, where would be a good point to start learning? I would be interested in spending some time learning how the darwin stdenv works, but every time I go looking for resources I don't find anything helpful.

While more documentation is always helpful, what is there right now that might be helpful?

[copumpkin]

@nixy sadly there's not much beyond "read the code" right now. In the meantime, here's a high-level overview that I should flesh out and stick into pkgs/stdenv/darwin/readme.md or something:

The darwin stdenv is built primarily from a combination of opensource.apple.com official Apple source releases, except when a pre-existing Nix package already handles the functionality in a smooth manner. The goal is to retain as much purity as possible, and also to be as similar as possible to the Linux stdenv to minimize the number of conditional behavior in downstream packages (i.e., I don't want every package to have an if stdenv.isDarwin then ... else ... in it). A consequence of these goals is that we don't use the BSD utilities in our stdenv, but instead stick to the GNU coreutils that we use elsewhere in Nix because they have slightly different combinations of command-line parameters and would result in more complicated conditional builds down the line. On the other hand, we do use Apple's xnu (kernel) and libc headers because those are the only ones that work on the platform.

The effort would be far simpler than it is today if these weren't true:

1. Apple's source releases almost never build outside of Apple. Either they contain redacted code (some parts of xnu, dyld, etc.) or Apple just doesn't care about ensuring that their stuff builds outside. There's an existing effort to get many of them to build by Sam Marshall, at https://github.com/samdmarshall/OSXPrivateSDK, but we don't use that directly because they don't really track where components of the private SDK came from and it's fairly monolithic (just like all Apple "SDKs").
2. Even if you can coax Apple source releases to build, they often rely on tools that have no public source available, and can't be downloaded easily, like xcodebuild. For this reason, we sometimes build some of the Apple source releases ourselves (i.e., invoke cc directly a bunch of times on each source file). That's obviously not sustainable but nobody's gotten around to plugging in a proper open source build tool yet, and we don't want the whole stdenv to depend on Xcode.

Because building a lot of these releases is either painful or very difficult, we often skip a complete build and instead cobble together headers from the packages. You'll find that the xnu and Libc builds work that way, for example (although I do have an unreleased xnu expression that builds a full kernel), but the goal is to get more of them to build directly rather than produce a set of hacky headers, because it'll improve purity. Unfortunately some of the builds rely on unreleased Apple projects like their "platform expert" project or launchd, which they stopped releasing after 10.10. Because of that, there are occasional hacks in the builds that pull in headers from older Apple releases (they never take anything down, so if they used to release launchd, we just pull in that header). We also submit radars asking that they release more code, but they're usually ignored.

The majority of the Apple-specific stdenv code is in https://github.com/NixOS/nixpkgs/blob/master/pkgs/stdenv/darwin/default.nix or in https://github.com/NixOS/nixpkgs/blob/master/pkgs/os-specific/darwin/apple-source-releases/default.nix. Also note that I recently made a few changes in staging so you might want to look at that branch instead.

To be covered: how we handle stuff that needs Apple-specific libraries frameworks, where/why impurity creeps in, how the sandbox fits in, and my vision of the general direction we should be heading in.

Actually impure things today:

1. All frameworks except for some limited uses of SystemConfiguration.framework and CoreFoundation.framework
2. libSystem.dylib (this can be reduced and possibly eliminated eventually with careful building)
3. dyld. This might have been reduced/eliminated prior to SIP, but now the xnu kernel hardcodes the expected path to dyld and makes it impossible for us to use a custom one. There might be workarounds, but it seems unlikely.

[periklis]

@copumpkin Thx for this some kind in depth introduction to stdenv.Darwin. First of all thanks for stepping up and making this as-pure-as-possible stdenv possible for us. Besides, just my 2 cents of my so far understanding of stdenv.darwin (Please correct me!):

1. Most packages in my 15-years osx-experience (brew, macports, etc.) need just a proper set of gnu *utils, mostly a decent apple-working compiler (clang - yes we can! on stdenv.darwin!), binutils and some fit of env vars. Beyond that the packages build script have to play nice with dynamic linking, which is the only less painful way on OSX. I think your work on stdenv covers already this stuff very well, right? do you see any rough edges or hardcoded stuff to help out?
2. Grepping a little bit over the repo tells me that we already patch-work with stdenv.isDarwin in 400 places over ~7630 nix files (just in nix-code, not sure if package shell builders do more). Do you think that this is too much/less ROI of the hypothesis to minimize conditional expressions? If too much, maybe enforcing conditional further?
3. After an intensive study of stdenv/darwin/default.nix it tells me that it is clear straight forward except of this stageFun recursion. However, i am sure its not only because of code-duplication-avoidance. I think that if you could elaborate in a couple of lines readme.md the stages' rationale and deps, anyone could step in and help out. I think the deps and their order is the highest mystery of stdenv for simple package maintainers on darwin. Anyone maintaining a package should easily understand this. What do you think?

@dhess & @copumpkin Imho the state of the nix-osx is that the stdenv is in a good state according to my current reading of stdenv/darwin/default.nix. However, maintaining packages on OSX written for linux or other unixes is a moving target by itself. If you have a look on the Formulas on homebrew, you will understand what i mean. Although, homebrew uses the impure parts of OSX, it stills builds the packages according to their build scripts and not xcodebuild. In my experience the Formula-maintainers have the same issues and patch explicitly, too. Despite the fact that the count of nix-osx contributors is small today, we can learn from the Formulas on homebrew in order to fix our packages. In my current opinion, i can't see how the current purity of stdenv.darwin constraints/limits package building in comparison to the impurity of homebrew. The only rough corner that i identify is building packages that depend on apple frameworks or opensource packages, which we rebuilt and put in nix-store and homebrew does not. So let me state the "State-Of-Nix-OSX"-question differently and let me know what you think of:

1. How can we learn of other osx package repositories in order to build linux/unix packages on osx with nix's stdenv.darwin?
2. How can we enable the non-darwin package maintainers to understand osx-specificities in order to minimize breakages on osx (hey, we are the minority, lets pull in the majority ;))?
3. How can we establish technical confidence in the small nix-osx contributors group that stdenv.darwin is not pulling any magic into the package building/linking but just covers the purity and puts some basic vars in place. Let's clear out which/how/why in a small readme.md?
4. Let's collect already good practices in packages building on nix-osx in order to help other package maintainers. Sometimes it's just a recurring buildInput, another time a compile/link flag or when to go for an osx specific builder.sh or even expression (e.g. emacs24macport). Another example is the OSX specificity using the link tools (e.g. libtool, install_name_tool, etc.).

[acowley]

There is so much in nix Darwin that works that it makes the rough spots that much more jarring. The choice to use the GNU tools works out very well for maximizing compatibility with Linux build scripts, as an example of a design choice that pays dividends. The problems I run into on a regular basis revolve solely around frameworks, with CoreFoundation being the most frequent trouble maker.

What's problematic there is that I do not see the larger goals of our exertions. With the GNU tools, we drastically cut down the need for Darwin-specific scripting, but once a build is reaching out for frameworks we've entered the bailiwick of the Darwin-specific build tools. These other tools, be they brew or just some cmake tooling, are built to accommodate standard Darwin build environments, i.e. what Apple provides. So I'd like to better understand the benefits of maintaining independence from Xcode: is nix better able to support a range of OS X releases? Is this critical for binary substitutes to work without building more macOS awareness into everything? For the sake of my own ability to get work done, I've taken of late to bypassing the nixpkgs frameworks when possible (e.g. glfw wants a newer OpenGL that 10.9's), and I am waiting for the other shoe to drop.

EDIT: I can offer at least a partial answer to my question. Xcode 8 includes the 10.12 SDK, header files from which can't be read by clang <= 3.8 (I haven't actually gotten it to work at all, but I hope 3.9 works!). So relying on the SDK Xcode installs would necessitate that the compiler be kept very current. I don't think the current approach of using a several-years-old SDK is particularly good, but there are clearly plenty of drawbacks to go around.

[meiersi-da]

Thanks to everybody who contributed to this very informative thread! I'm quite new to nix on OS X, so forgive me if the following question is dumb. Why don't we reflect the SDK version into the system argument, and consider different releases of OS X to be different operating systems -- at least on a logical level? It seems to me that this would be a (the only?) faithful model that allows us to use the OS X build tools in a logically pure fashion.

Given this increased precision, I would then expect that the following things become possible.

1. Packages can specify lower-bound conditions on the SDK for which they can be built.
2. When building a package on OS X, I can specify the target SDK version, and the stdenv appropriately instructs xcodebuild on what SDK should be used. This way I can still build packages for older OS X's on a newer build machine.

Thanks to Point 2., one should be able to offer a binary cache for a specific minimal version of OS X without introducing additional compilation overhead. I'm thinking here in particular about the use-case that we have in our company where we distribute developer tools for OS X via nix.

To me, the proposed way above seems to be not as nice as having a stdenv with all frameworks built from source. However, given the trouble of building such a stdenv with the code Apple open sources, it seems to me that it would be a nice pragmatic tradeoff with a good semantics. What am I overlooking? How difficult would this to be to implement for nixpkgs?

[copumpkin]

@meiersi-da in short, because we don't use the Apple SDK, xcodebuild, or any of that machinery, or even depend on Xcode or the command-line tools being installed. Depending on external state like SDK version decreases overall purity and makes our Hydra infrastructure more complex, so we'd rather not until we can't avoid it.

[meiersi-da]

OK, that does make sense. Thanks for the explanation, and all the effort that you are putting into this. It already works very well, and I'm looking forward to the improvements that you were talking about. I'll try to free some resources to give back.

[acowley]

For anyone else who is completely stuck due to the age of the SDK in nixpkgs, here is a definition one can use to depend on a versioned macOS SDK. You use this in your buildInputs rather than the frameworks in nixpkgs.

[meiersi-da]

Thanks for sharing. That might come handy indeed.

[dhess]

Here's something I don't get, and maybe @copumpkin or someone else can provide the rationale.

Pretty much anything in nixpkgs that uses autoconf to build, and clock_gettime and friends at compile-time, breaks on macOS 10.12. Apple added clock_gettime functionality in 10.12, so the dylibs support it, but the headers being pulled in from various old pre-10.12 source releases do not. So if, for example, your autoconf stage tries to build a clock_gettime example program (presumably they extern the symbol in the example program?), it will show up in config.h but then won't compile in the actual build stage.

Why does Darwin stdenv symlink the system's dylibs, but then use the header files from a downloaded source release?

This is a major show-stopper right now as pretty much anything that uses glib won't compile on 10.12. You can't even build nix at the moment, neither in staging nor in master.

(edit: ahh, re-reading the thread above, are the include files not installed unless you've installed the Xcode command-line utilities?)

[copumpkin]

I'm traveling until Sunday, then I should be able to deal with all this stuff and answer actual questions. Sorry!

[meiersi-da]

(No hurry in my case. I really do appreciate all your work on this a lot! Safe travels 🍻 )

[ghost]

Pretty much anything in nixpkgs that uses autoconf to build, and clock_gettime and friends at compile-time, breaks on macOS 10.12

Yes, this is really sad.

[copumpkin]

We'll fix it folks. We've had a similar issue before and just need to reinstate a thing that we did a while ago to make it work. Sorry I've been absent for so long but I'll actually be home for the first time in a while this weekend and can start getting this sierra thing back under control.

On Oct 7, 2016, at 03:23, gnidorah notifications@github.com wrote:

Pretty much anything in nixpkgs that uses autoconf to build, and clock_gettime and friends at compile-time, breaks on macOS 10.12

Yes, this is really sad.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[copumpkin]

@dhess in short, we impurely link to the libSystem.dylib because we have to, and use a fixed set of headers because we can. One day perhaps we'll be able to build our own libSystem (or at least the libc constituents of it) but in the meantime we've attempted to maximize purity by downloading a fixed set of headers and then linking against a subset of libsystem. More recently, I removed that "subset" code, but I think I'll have to put it back due to these changes. I'm looking into that now.

[copumpkin]

(edit: ahh, re-reading the thread above, are the include files not installed unless you've installed the Xcode command-line utilities?)

Also, no, nothing in the usual stdenv should depend on anything external like Xcode or the command-line utilities. The only impure dependencies in the stdenv are libsystem, dyld, /bin/sh, and /usr/bin/env.

[yurrriq]

I'm not sure if this has been mentioned around here yet, but I've been using @LnL7's nix-darwin for nearly three weeks now, and its development is progressing nicely.

[dhess]

Has anyone tried Nix on the macOS High Sierra beta yet? It would be nice to get ahead of the final release this time, if possible.

[copumpkin]

Yeah, fixes for it are already in master but they're rather hard to test until 17.09 is cut. If you want to try installing try this script: https://gist.github.com/copumpkin/8373ae6c89236989ae4296771ba5aa14

[dhess]

Cool, thanks! I don't have access to a spare Mac at the moment, but I'll try it out once I do, hopefully within the next week or two.

[mjhoy]

Pretty much anything in nixpkgs that uses autoconf to build, and clock_gettime and friends at compile-time, breaks on macOS 10.12

Is there a fix for this? I seem to be running into this trying to compile GHC in a nix-shell on 10.12, on nixos-unstable.

[copumpkin]

@mjhoy are you on a recent nixpkgs? I haven't seen that issue in ages, and we should have fixed it last year

[mjhoy]

@copumpkin yes... just updated, 799435b7ca

hmm. I assumed it's related but maybe not. I get the following halfway through running make in ghc:

CTimespec.hsc:17:15: error: use of undeclared identifier 'clockid_t'
    hsc_type (clockid_t);
              ^

[copumpkin]

Oh, compiling in a nix-shell might be odd. The GHC package in nixpkgs works though, right?

[mjhoy]

Yes, the ghc binary works fine.

I am running nix-shell '<nixpkgs>' -A haskell.compiler.ghcHEAD to build. Should I open a separate issue? Or am I missing something.

[copumpkin]

It's just that running compilers (GHC or otherwise) ourselves interactively doesn't always work well, compared to running them inside a full Nix builder environment. It might make sense to open a specific issue about it, yeah.

[dhess]

I'm now running the macOS High Sierra developer beta on all of my Macs, and I'm happy to report that my Nix environment is working fine so far, including the usual problematic suspects such as GHC.

FYI I'm on the nixpkgs-channels repo, nixpkgs-unstable branch (as of commit 017561209e933f3de086e04211dc48cf37d1ee82); and this isn't a fresh installation of either macOS High Sierra or Nix. Nix was already installed and these Macs were upgraded from macOS Sierra.

I will post an update again in a few days when I'm brave enough to update my repo and do some actual Nix builds in High Sierra. In any case, this is a huge improvement vs. previous macOS releases. Thanks to @copumpkin and whoever else contributed to the High Sierra support! Long may it last.

[copumpkin]

Great to hear! The official Nix builds on Hydra should also be compatible with 10.13 now, if you want to stop using my dodgy installer 😄

[dhess]

I didn't use your installer, actually; just updated from Hydra/the cache as I usually do. No changes were required at all in my Nix environment.

[dhess]

This issue has been open for just over a year now, and things are so much better these days on macOS! I think this issue can be closed. Thank you to @copumpkin, @LnL7, and @grahamc for your hard work on improving macOS support in Nix. (And to anyone I missed who has made a significant contribution, I am sorry! This list is not meant to be exclusive, I am just too lazy to search the commit logs to find all those who helped.)

Before I close it, can we get an update on sandbox support in macOS? I have seen some indication that it's working again, but maybe only with the upcoming nix-1.12 release?

[copumpkin]

@dhess sandbox support is coming back in 1.12. @LnL7 and I put in a decent amount of work around NixCon to bring it back to a good spot, but since then I haven't had much time for Nix.

Assuming all goes to plan (and 1.12 is held up for a bit longer, given my lack of time to engage), we'd like 1.12 to be released pointing to a stable macOS nixpkgs channel, with sandboxing turned on by default on our platform. We might not be able to pull off all of it, but even if we default sandboxing to on and don't point at the stable channel by default, things would mostly work in the new release.

[dhess]

@copumpkin Thank you for the update!