Failed to cross compile arm target on x86 mac

[houqp]

Hi,

I am trying to cross compile to an arm binary from a x86 mac machine in github action job, but it's failing with the following error:

  /usr/local/Cellar/cmake/3.21.3/bin/cmake -E cmake_progress_start /Users/runner/work/roapi/roapi/target/aarch64-apple-darwin/debug/build/snmalloc-sys-77e000b0b98622f1/out/build/CMakeFiles 1
  /Applications/Xcode_12.4.app/Contents/Developer/usr/bin/make  -f CMakeFiles/Makefile2 CMakeFiles/snmallocshim-1mib-rust.dir/all
  /Applications/Xcode_12.4.app/Contents/Developer/usr/bin/make  -f CMakeFiles/snmallocshim-1mib-rust.dir/build.make CMakeFiles/snmallocshim-1mib-rust.dir/depend
  cd /Users/runner/work/roapi/roapi/target/aarch64-apple-darwin/debug/build/snmalloc-sys-77e000b0b98622f1/out/build && /usr/local/Cellar/cmake/3.21.3/bin/cmake -E cmake_depends "Unix Makefiles" /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc /Users/runner/work/roapi/roapi/target/aarch64-apple-darwin/debug/build/snmalloc-sys-77e000b0b98622f1/out/build /Users/runner/work/roapi/roapi/target/aarch64-apple-darwin/debug/build/snmalloc-sys-77e000b0b98622f1/out/build /Users/runner/work/roapi/roapi/target/aarch64-apple-darwin/debug/build/snmalloc-sys-77e000b0b98622f1/out/build/CMakeFiles/snmallocshim-1mib-rust.dir/DependInfo.cmake --color=
  /Applications/Xcode_12.4.app/Contents/Developer/usr/bin/make  -f CMakeFiles/snmallocshim-1mib-rust.dir/build.make CMakeFiles/snmallocshim-1mib-rust.dir/build
  [100%] Building CXX object CMakeFiles/snmallocshim-1mib-rust.dir/src/override/rust.cc.o
  /usr/bin/c++ -DBACKTRACE_HEADER=\"execinfo.h\" -DCACHE_FRIENDLY_OFFSET=64 -DSNMALLOC_EXPORT="__attribute__((visibility(\"default\")))" -I/Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/src -ffunction-sections -fdata-sections -fPIC -arch arm64 -O3 -DNDEBUG -isysroot /Applications/Xcode_12.4.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX11.1.sdk -mmacosx-version-min=10.9 -fvisibility=hidden -Wsign-conversion -Wconversion -Wall -Wextra -Werror -Wundef -fno-exceptions -fno-rtti -g -fomit-frame-pointer -ftls-model=initial-exec -mcx16 -std=gnu++17 -MD -MT CMakeFiles/snmallocshim-1mib-rust.dir/src/override/rust.cc.o -MF CMakeFiles/snmallocshim-1mib-rust.dir/src/override/rust.cc.o.d -o CMakeFiles/snmallocshim-1mib-rust.dir/src/override/rust.cc.o -c /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/src/override/rust.cc

  --- stderr
  CMake Warning (dev) at /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCompiler.cmake:143 (set):
    Policy CMP0126 is not set: set(CACHE) does not remove a normal variable of
    the same name.  Run "cmake --help-policy CMP0126" for policy details.  Use
    the cmake_policy command to set the policy and suppress this warning.

    For compatibility with older versions of CMake, normal variable
    "CMAKE_C_COMPILER" will be removed from the current scope.
  Call Stack (most recent call first):
    /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCCompiler.cmake:68 (_cmake_find_compiler_path)
    CMakeLists.txt:2 (project)
  This warning is for project developers.  Use -Wno-dev to suppress it.

     Called from: [3]   /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCompiler.cmake
                  [2]   /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCCompiler.cmake
                  [1]   /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/CMakeLists.txt
  CMake Warning (dev) at /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCompiler.cmake:143 (set):
    Policy CMP0126 is not set: set(CACHE) does not remove a normal variable of
    the same name.  Run "cmake --help-policy CMP0126" for policy details.  Use
    the cmake_policy command to set the policy and suppress this warning.

    For compatibility with older versions of CMake, normal variable
    "CMAKE_CXX_COMPILER" will be removed from the current scope.
  Call Stack (most recent call first):
    /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCXXCompiler.cmake:66 (_cmake_find_compiler_path)
    CMakeLists.txt:2 (project)
  This warning is for project developers.  Use -Wno-dev to suppress it.

     Called from: [3]   /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCompiler.cmake
                  [2]   /usr/local/Cellar/cmake/3.21.3/share/cmake/Modules/CMakeDetermineCXXCompiler.cmake
                  [1]   /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/CMakeLists.txt
  CMake Warning at CMakeLists.txt:79 (message):
    Not generating clangformat target, no clang-format tool found
  Call Stack (most recent call first):
    CMakeLists.txt:366 (clangformat_targets)

     Called from: [2]   /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/CMakeLists.txt
                  [1]   /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/snmalloc-sys-0.2.27/snmalloc/CMakeLists.txt
  CMake Warning:
    Manually-specified variables were not used by the project:

      CMAKE_ASM_COMPILER
      CMAKE_ASM_FLAGS

  clang: error: argument unused during compilation: '-mcx16' [-Werror,-Wunused-command-line-argument]
  make[3]: *** [CMakeFiles/snmallocshim-1mib-rust.dir/src/override/rust.cc.o] Error 1
  make[2]: *** [CMakeFiles/snmallocshim-1mib-rust.dir/all] Error 2
  make[1]: *** [CMakeFiles/snmallocshim-1mib-rust.dir/rule] Error 2
  make: *** [snmallocshim-1mib-rust] Error 2
  thread 'main' panicked at '
  command did not execute successfully, got: exit status: 2

Here is the build command I used to reproduce the issue:

cargo build --bin roapi-http --target aarch64-apple-darwin --features=simd 

I can see that the -mcx16 flag is set in snmalloc-rs/snmalloc-sys/build.rs. Is this a bug in the cc crate? It looks like the build.flag_if_supported("-mcx16") flag test is performed with the host target instead of the specified target.

Here is the host environment provided by github action:

Darwin Mac-1633928908187.local 19.6.0 Darwin Kernel Version 19.6.0: Tue Aug 24 20:28:00 PDT 2021; root:xnu-6153.141.40~1/RELEASE_X86_64 x86_64

Full CI run error can be found at: https://github.com/roapi/roapi/pull/89/checks?check_run_id=3855785363

GitHub action task config:

  mac_build:
    runs-on: macos-latest
    steps:
      - uses: actions/checkout@v2
      - name: Install Rust toolchain
        uses: actions-rs/toolchain@v1
        with:
          toolchain: nightly-${{ env.RUST_TC_NIGHTLY_VER }}
          target: aarch64-apple-darwin
          profile: minimal
          default: true
      - name: Build binary
        env:
          DEVELOPER_DIR: /Applications/Xcode.app/Contents/Developer
          MACOSX_DEPLOYMENT_TARGET: '10.9'
        run: |
          # set SDKROOT for C dependencies
          export SDKROOT=$(xcrun --sdk macosx --show-sdk-path)
          cd roapi-http && \
            cargo build --bin roapi-http --target aarch64-apple-darwin --features=simd

[mjp41]

So I have been looking at debugging this. The current master doesn't seem to have an issue with cross build. I set up a build on Github actions, that does the same cross build you had:

163

The additional job builds successfully. Could you try 0.2.28? @SchrodingerZhu released it over the weekend.

[SchrodingerZhu]

FYI, after 0.2.28 where we sync a change to let cmake automatically detect mcx16, the cross building should be easier now.

[houqp]

oh nice, let me give 0.2.28 a try! Well timed on the release :D

[houqp]

Can confirm 0.2.28 fixed the build issue! Thank you all for the help. snmalloc is really awesome, I will try to push more adoption of it at work :D

[mjp41]

Can confirm 0.2.28 fixed the build issue! Thank you all for the help.

Great to hear it is working.

snmalloc is really awesome, I will try to push more adoption of it at work :D

Thanks, it's great to hear it is working well for you.