sandboxed builds with many sandbox rules fail on Darwin

mroi commented 6 years ago

I am using Nix on macOS and I have sandboxed builds enabled. When I build texlive derivations using the texlive.combine function, the build can fail with this message:

these derivations will be built:
  /nix/store/1sjfz02piyqgplvmw8xdxz8gfaka1brj-texlive-combined-2017.drv
building '/nix/store/1sjfz02piyqgplvmw8xdxz8gfaka1brj-texlive-combined-2017.drv'...
Assertion failed: (diff <= INSTR_JUMP_NE_MAX_LENGTH), function push_jne_instr, file /BuildRoot/Library/Caches/com.apple.xbs/Sources/Sandbox/Sandbox-851.200.128.41.1/src/compiler/patterns/serialize.c, line 186.
builder for '/nix/store/1sjfz02piyqgplvmw8xdxz8gfaka1brj-texlive-combined-2017.drv' failed due to signal 6 (Abort trap: 6)
error: build of '/nix/store/1sjfz02piyqgplvmw8xdxz8gfaka1brj-texlive-combined-2017.drv' failed

My guess is that this is caused by the sandbox profile containing too many rules. Disabling sandboxing and restarting nix-daemon causes the build to succeed. This is also currently my workaround: disable sandboxing whenever something fails like this, then reenable.

A better solution would probably be to check the length of the sandbox profile (somewhere around here in build.cc I guess) and fall back to a simpler profile automatically. How do you guys feel about this? I am willing to help getting this solved.

edolstra commented 6 years ago

Ping @copumpkin, @LnL7.

copumpkin commented 6 years ago

Wondering if this would still happen if we stopped doing as much "codegen" and started moving towards a more "data + code" sandbox design, as I've talked about in the past. That is, we'd factor out the common sandbox patterns into scheme on the sandbox side, rather than C++ on the Nix side.

My understanding is that the scheme translates to a sort of bytecode that then gets sent to the kernel, and there's a branch destination limit in the encoding of that bytecode, so jumps that are too long fail as above. If there isn't all that much actual code that would stop the error from arising.

lheckemann commented 6 years ago

Disabling sandboxing and restarting nix-daemon causes the build to succeed

As an easier workaround (but a workaround nonetheless) I think you should be able to pass --no-sandbox on the command line as long as you're root or listed in trusted-users.

mroi commented 6 years ago

Ah, did not know that. Thanks for the hint.

angerman commented 4 years ago

Just ran into this today. Could run nix-build with --option sandbox true and build it successfully nor with --option sandbox false, due to some cc tools confusion.

Ended up doing --option sandbox true and then nix-build those failing derivations by hand. Not idea. If anyone iwth some more insight has a TODO list of items to be done to get sandboxes working properly on macOS, let me know!