boost: cross compile from x86_64-darwin to aarch64-darwin is broken

[jfchevrette]

Describe the bug

When cross compiling boost from x86_64-darwin to aarch64-darwin (on a M1 mac) the compilation fails with the following error

/private/tmp/nix-build-boost-aarch64-apple-darwin-1.69.0.drv-0/boost_1_69_0/tools/build/src/build/feature.jam:491: in validate-value-string from module feature
error: "macho" is not a known value of feature <binary-format>
error: legal values: "elf" "mach-o" "pe" "xcoff"

One fix I attempted is the following:

-    "binary-format=${toString stdenv.hostPlatform.parsed.kernel.execFormat.name}"
+    "binary-format=${if toString stdenv.hostPlatform.parsed.kernel.execFormat.name == "macho" then "mach-o"
+                     else toString stdenv.hostPlatform.parsed.kernel.execFormat.name}"

Which worked. Unfortunately the compilation fails further for a different reason:

clang-darwin.compile.asm bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/make_arm64_aapcs_macho_gas.o
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:59:21: error: unexpected token in argument list
    ; shift address in x0 (allocated stack) to lower 16 byte boundary
                    ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:60:5: error: unknown use of instruction mnemonic without a size suffix
    and x0, x0, ~0xF
    ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:62:21: error: unexpected token in argument list
    ; reserve space for context-data on context-stack
                    ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:63:17: error: unknown token in expression
    sub x0, x0, #0xb0
                ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:65:17: error: unexpected token in argument list
    ; third arg of make_fcontext() == address of context-function
                ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:66:21: error: unexpected token in argument list
    ; store address as a PC to jump in
                    ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:67:13: error: brackets expression not supported on this target
    str x2, [x0, #0xa0]
            ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:69:19: error: unexpected token in argument list
    ; compute abs address of label finish
                  ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:70:7: error: unexpected token at start of statement
    ; 0x0c = 3 instructions * size (4) before label 'finish'
      ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:72:28: error: unexpected token in argument list
    ; TODO: Numeric offset since llvm still does not support labels in ADR. Fix:
                           ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:74:5: error: invalid instruction mnemonic 'adr'
    adr x1, 0x0c
    ^~~
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:76:20: error: unexpected token in argument list
    ; save address of finish as return-address for context-function
                   ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:77:15: error: unexpected token in argument list
    ; will be entered after context-function returns (LR register)
              ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:78:13: error: brackets expression not supported on this target
    str x1, [x0, #0x98]
            ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:80:5: error: unknown use of instruction mnemonic without a size suffix
    ret lr ; return pointer to context-data (x0)
    ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:80:29: error: unexpected token in argument list
    ret lr ; return pointer to context-data (x0)
                            ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:83:17: error: unexpected token in argument list
    ; exit code is zero
                ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:84:13: error: unknown token in expression
    mov x0, #0
            ^
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:85:7: error: invalid instruction mnemonic 'exit'
    ; exit application
      ^~~~
libs/context/src/asm/make_arm64_aapcs_macho_gas.S:86:5: error: invalid instruction mnemonic 'bl'
    bl __exit
    ^~

    "clang++" -x assembler-with-cpp -fPIC -O3 -Wall -fvisibility=hidden -Wno-inline  -DBOOST_ALL_NO_LIB=1 -DBOOST_CONTEXT_DYN_LINK=1 -DBOOST_CONTEXT_SOURCE -DBOOST_DISABLE_ASSERTS -DNDEBUG -I"." -c -o "bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/make_arm64_aapcs_macho_gas.o" "libs/context/src/asm/make_arm64_aapcs_macho_gas.S"

...failed clang-darwin.compile.asm bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/make_arm64_aapcs_macho_gas.o...
clang-darwin.compile.asm bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/jump_arm64_aapcs_macho_gas.o
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:58:21: error: unexpected token in argument list
    ; prepare stack for GP + FPU
                    ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:59:17: error: unknown token in expression
    sub sp, sp, #0xb0
                ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:61:7: error: invalid instruction mnemonic 'save'
    ; save d8 - d15
      ^~~~
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:62:17: error: brackets expression not supported on this target
    stp d8, d9, [sp, #0x00]
                ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:63:19: error: brackets expression not supported on this target
    stp d10, d11, [sp, #0x10]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:64:19: error: brackets expression not supported on this target
    stp d12, d13, [sp, #0x20]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:65:19: error: brackets expression not supported on this target
    stp d14, d15, [sp, #0x30]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:67:7: error: invalid instruction mnemonic 'save'
    ; save x19-x30
      ^~~~
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:68:19: error: brackets expression not supported on this target
    stp x19, x20, [sp, #0x40]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:69:19: error: brackets expression not supported on this target
    stp x21, x22, [sp, #0x50]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:70:19: error: brackets expression not supported on this target
    stp x23, x24, [sp, #0x60]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:71:19: error: brackets expression not supported on this target
    stp x25, x26, [sp, #0x70]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:72:19: error: brackets expression not supported on this target
    stp x27, x28, [sp, #0x80]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:73:17: error: brackets expression not supported on this target
    stp fp, lr, [sp, #0x90]
                ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:75:15: error: unexpected token in argument list
    ; save LR as PC
              ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:76:13: error: brackets expression not supported on this target
    str lr, [sp, #0xa0]
            ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:78:18: error: unexpected token in memory operand
    ; store RSP (pointing to context-data) in X0
                 ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:79:5: error: unknown use of instruction mnemonic without a size suffix
    mov x4, sp
    ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:81:20: error: unexpected token in memory operand
    ; restore RSP (pointing to context-data) from X1
                   ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:82:5: error: unknown use of instruction mnemonic without a size suffix
    mov sp, x0
    ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:84:7: error: invalid instruction mnemonic 'load'
    ; load d8 - d15
      ^~~~
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:85:17: error: brackets expression not supported on this target
    ldp d8, d9, [sp, #0x00]
                ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:86:19: error: brackets expression not supported on this target
    ldp d10, d11, [sp, #0x10]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:87:19: error: brackets expression not supported on this target
    ldp d12, d13, [sp, #0x20]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:88:19: error: brackets expression not supported on this target
    ldp d14, d15, [sp, #0x30]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:90:7: error: invalid instruction mnemonic 'load'
    ; load x19-x30
      ^~~~
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:91:19: error: brackets expression not supported on this target
    ldp x19, x20, [sp, #0x40]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:92:19: error: brackets expression not supported on this target
    ldp x21, x22, [sp, #0x50]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:93:19: error: brackets expression not supported on this target
    ldp x23, x24, [sp, #0x60]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:94:19: error: brackets expression not supported on this target
    ldp x25, x26, [sp, #0x70]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:95:19: error: brackets expression not supported on this target
    ldp x27, x28, [sp, #0x80]
                  ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:96:17: error: brackets expression not supported on this target
    ldp fp, lr, [sp, #0x90]
                ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:98:25: error: unexpected token in argument list
    ; return transfer_t from jump
                        ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:99:23: error: unexpected token in argument list
    ; pass transfer_t as first arg in context function
                      ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:100:10: error: unknown token in expression
    ; X0 == FCTX, X1 == DATA
         ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:101:5: error: unknown use of instruction mnemonic without a size suffix
    mov x0, x4
    ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:103:7: error: invalid instruction mnemonic 'load'
    ; load pc
      ^~~~
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:104:13: error: brackets expression not supported on this target
    ldr x4, [sp, #0xa0]
            ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:106:21: error: unexpected token in argument list
    ; restore stack from GP + FPU
                    ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:107:17: error: unknown token in expression
    add sp, sp, #0xb0
                ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:109:5: error: unknown use of instruction mnemonic without a size suffix
    ret x4
    ^
libs/context/src/asm/jump_arm64_aapcs_macho_gas.S:110:1: error: assembler local symbol 'LR' not defined

^

    "clang++" -x assembler-with-cpp -fPIC -O3 -Wall -fvisibility=hidden -Wno-inline  -DBOOST_ALL_NO_LIB=1 -DBOOST_CONTEXT_DYN_LINK=1 -DBOOST_CONTEXT_SOURCE -DBOOST_DISABLE_ASSERTS -DNDEBUG -I"." -c -o "bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/jump_arm64_aapcs_macho_gas.o" "libs/context/src/asm/jump_arm64_aapcs_macho_gas.S"

...failed clang-darwin.compile.asm bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/jump_arm64_aapcs_macho_gas.o...
clang-darwin.compile.c++ bin.v2/libs/container/build/clang-darwin-7.1/release/architecture-arm/threading-multi/visibility-hidden/dlmalloc.o
clang-darwin.compile.c++ bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/execution_context.o
clang-darwin.compile.c++ bin.v2/libs/thread/build/clang-darwin-7.1/release/architecture-arm/threadapi-pthread/threading-multi/visibility-hidden/pthread/once.o
clang-darwin.compile.c++ bin.v2/libs/container/build/clang-darwin-7.1/release/architecture-arm/threading-multi/visibility-hidden/global_resource.o
clang-darwin.compile.c bin.v2/libs/container/build/clang-darwin-7.1/release/architecture-arm/threading-multi/visibility-hidden/alloc_lib.o
clang-darwin.compile.c++ bin.v2/libs/thread/build/clang-darwin-7.1/release/architecture-arm/threadapi-pthread/threading-multi/visibility-hidden/future.o
clang-darwin.compile.asm bin.v2/libs/context/build/clang-darwin-7.1/release/abi-aapcs/architecture-arm/threading-multi/visibility-hidden/asm/ontop_arm64_aapcs_macho_gas.o
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:58:21: error: unexpected token in argument list
    ; prepare stack for GP + FPU
                    ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:59:17: error: unknown token in expression
    sub sp, sp, #0xb0
                ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:61:7: error: invalid instruction mnemonic 'save'
    ; save d8 - d15
      ^~~~
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:62:17: error: brackets expression not supported on this target
    stp d8, d9, [sp, #0x00]
                ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:63:19: error: brackets expression not supported on this target
    stp d10, d11, [sp, #0x10]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:64:19: error: brackets expression not supported on this target
    stp d12, d13, [sp, #0x20]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:65:19: error: brackets expression not supported on this target
    stp d14, d15, [sp, #0x30]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:67:7: error: invalid instruction mnemonic 'save'
    ; save x19-x30
      ^~~~
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:68:19: error: brackets expression not supported on this target
    stp x19, x20, [sp, #0x40]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:69:19: error: brackets expression not supported on this target
    stp x21, x22, [sp, #0x50]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:70:19: error: brackets expression not supported on this target
    stp x23, x24, [sp, #0x60]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:71:19: error: brackets expression not supported on this target
    stp x25, x26, [sp, #0x70]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:72:19: error: brackets expression not supported on this target
    stp x27, x28, [sp, #0x80]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:73:19: error: brackets expression not supported on this target
    stp x29, x30, [sp, #0x90]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:75:15: error: unexpected token in argument list
    ; save LR as PC
              ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:76:14: error: brackets expression not supported on this target
    str x30, [sp, #0xa0]
             ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:78:18: error: unexpected token in memory operand
    ; store RSP (pointing to context-data) in X5
                 ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:79:5: error: unknown use of instruction mnemonic without a size suffix
    mov x4, sp
    ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:81:20: error: unexpected token in memory operand
    ; restore RSP (pointing to context-data) from X1
                   ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:82:5: error: unknown use of instruction mnemonic without a size suffix
    mov sp, x0
    ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:84:7: error: invalid instruction mnemonic 'load'
    ; load d8 - d15
      ^~~~
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:85:17: error: brackets expression not supported on this target
    ldp d8, d9, [sp, #0x00]
                ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:86:19: error: brackets expression not supported on this target
    ldp d10, d11, [sp, #0x10]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:87:19: error: brackets expression not supported on this target
    ldp d12, d13, [sp, #0x20]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:88:19: error: brackets expression not supported on this target
    ldp d14, d15, [sp, #0x30]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:90:7: error: invalid instruction mnemonic 'load'
    ; load x19-x30
      ^~~~
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:91:19: error: brackets expression not supported on this target
    ldp x19, x20, [sp, #0x40]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:92:19: error: brackets expression not supported on this target
    ldp x21, x22, [sp, #0x50]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:93:19: error: brackets expression not supported on this target
    ldp x23, x24, [sp, #0x60]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:94:19: error: brackets expression not supported on this target
    ldp x25, x26, [sp, #0x70]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:95:19: error: brackets expression not supported on this target
    ldp x27, x28, [sp, #0x80]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:96:19: error: brackets expression not supported on this target
    ldp x29, x30, [sp, #0x90]
                  ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:98:25: error: unexpected token in argument list
    ; return transfer_t from jump
                        ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:99:23: error: unexpected token in argument list
    ; pass transfer_t as first arg in context function
                      ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:100:10: error: unknown token in expression
    ; X0 == FCTX, X1 == DATA
         ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:101:5: error: unknown use of instruction mnemonic without a size suffix
    mov x0, x4
    ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:103:7: error: invalid instruction mnemonic 'skip'
    ; skip pc
      ^~~~
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:104:21: error: unexpected token in argument list
    ; restore stack from GP + FPU
                    ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:105:17: error: unknown token in expression
    add sp, sp, #0xb0
                ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:107:15: error: unexpected token in argument list
    ; jump to ontop-function
              ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:108:5: error: unknown use of instruction mnemonic without a size suffix
    ret x2
    ^
libs/context/src/asm/ontop_arm64_aapcs_macho_gas.S:109:1: error: assembler local symbol 'LR' not defined

Both boost16x and boost17x produces the same errors during compilation. I am not sure if this is an issue Nix should be fixing or if this should be reported upstream

To Reproduce

nix-repl> stdenv.hostPlatform.system
"x86_64-darwin"

nix-repl> stdenv.hostPlatform.parsed.kernel.execFormat.name
"macho"

$ nix-build '<nixpkgs>' -A pkgsCross.aarch64-darwin.boost
$ nix-build '<nixpkgs>' -A pkgsCross.aarch64-darwin.boost16x
$ nix-build '<nixpkgs>' -A pkgsCross.aarch64-darwin.boost17x

Expected behavior boost should compile

Notify maintainers @peti

Metadata

nix-shell -p nix-info --run "nix-info -m"
 - system: `"x86_64-darwin"`
 - host os: `Darwin 20.5.0, macOS 10.16`
 - multi-user?: `no`
 - sandbox: `no`
 - version: `nix-env (Nix) 2.4pre20210601_5985b8b`
 - channels(jfchevrette): `"nixpkgs-21.11pre295670.189a1368878"`
 - nixpkgs: `/Users/jfchevrette/.nix-defexpr/channels/nixpkgs`

Maintainer information:

# a list of nixpkgs attributes affected by the problem
attribute:
# a list of nixos modules affected by the problem
module:

[redbaron]

Looks like b2 doesn't pass arch flag: https://github.com/bfgroup/b2/issues/136

[Thesola10]

looks like upstream is fixed, also the macho vs mach-o would be nice for #214611