otool killed during post-processing on osx-arm64

[mfansler]

Actual Behavior

Running conda build --R 4.1 . results in unexpected exit during the post-processing stage of packaging when running on native osx-arm64. Below is the traceback, including some of the preceding "INFO" outputs to establish context. Note that termination occurs in the otool -l call, and right before the stack trace is a message from libc++abi that observed the exception codesign_failed: 0.

Packaging r-rcpp
INFO:conda_build.build:Packaging r-rcpp
INFO conda_build.build:build(2294): Packaging r-rcpp
Packaging r-rcpp-1.0.8-r41h95ea553_1
INFO:conda_build.build:Packaging r-rcpp-1.0.8-r41h95ea553_1
INFO conda_build.build:bundle_conda(1534): Packaging r-rcpp-1.0.8-r41h95ea553_1
number of files: 653
Fixing linking of Rcpp.dylib in /Users/mfansler/mambaforge-arm64/conda-bld/r-rcpp_1646804920874/_h_env_placehold_placehold_placehold_pl\
acehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placeho\
ld_placehold_plac/lib/R/library/Rcpp/libs/Rcpp.dylib
New link location is lib/R/library/Rcpp/libs/Rcpp.dylib
[cctools-port]: generating fake signature for '/Users/mfansler/mambaforge-arm64/conda-bld/r-rcpp_1646804920874/_h_env_placehold_placeho\
ld_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pl\
acehold_placehold_placehold_plac/lib/R/library/Rcpp/libs/Rcpp.dylib'
libc++abi: terminating with uncaught exception of type std::runtime_error: codesign_failed: 0

Traceback (most recent call last):
  File "/Users/mfansler/mambaforge-arm64/bin/conda-mambabuild", line 10, in <module>
    sys.exit(main())
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/boa/cli/mambabuild.py", line 239, in main
    call_conda_build(action, config)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/boa/cli/mambabuild.py", line 211, in call_conda_build
    result = api.build(
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/api.py", line 186, in build
    return build_tree(
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/build.py", line 3088, in build_tree
    packages_from_this = build(metadata, stats,
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/build.py", line 2371, in build
    newly_built_packages = bundlers[pkg_type](output_d, m, env, stats)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/build.py", line 1664, in bundle_conda
    files = post_process_files(metadata, initial_files)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/build.py", line 1510, in post_process_files
    post_build(m, new_files, build_python=python)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/post.py", line 1317, in post_build
    post_process_shared_lib(m, f, prefix_files, host_prefix)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/post.py", line 1265, in post_process_shared_lib
    mk_relative_osx(path, host_prefix, m, files=files, rpaths=rpaths)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/post.py", line 409, in mk_relative_osx
    existing_rpaths = macho.get_rpaths(path, build_prefix=prefix)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/os_utils/macho.py", line 221, in get_rpaths
    dylib_loads = otool(path, build_prefix, is_rpath)
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/site-packages/conda_build/os_utils/macho.py", line 192, in otool
    lines = check_output([otool, '-l', path],
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/subprocess.py", line 424, in check_output
    return run(*popenargs, stdout=PIPE, timeout=timeout, check=True,
  File "/Users/mfansler/mambaforge-arm64/lib/python3.9/subprocess.py", line 528, in run
    raise CalledProcessError(retcode, process.args,
subprocess.CalledProcessError: Command '['/Users/mfansler/mambaforge-arm64/conda-bld/r-rcpp_1646804920874/_build_env/bin/arm64-apple-da\
rwin20.0.0-otool', '-l', '/Users/mfansler/mambaforge-arm64/conda-bld/r-rcpp_1646804920874/_h_env_placehold_placehold_placehold_placehol\
d_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_pla\
cehold_plac/lib/R/library/Rcpp/libs/Rcpp.dylib']' died with <Signals.SIGKILL: 9>.

Expected Behavior

It should not fail.

Steps to Reproduce

This seems to affect all R packages that involve C/C++ compilation that I have attempted so far. For example, with conda-forge/r-rcpp-feedstock

git clone https://github.com/conda-forge/r-rcpp-feedstock.git
cd r-rcpp-feedstock
conda build --R 4.1 recipe/.

Additional notes

Changing path into the residual working directory (under conda-bld/), activating the _build_env/ environment, and rerunning the failed command (essentially, arm64-apple-darwin20.0.0-otool -l Rcpp.dylib) also gets killed:

zsh: killed     arm64-apple-darwin20.0.0-otool -l

However, running the native otool -l (from Xcode) has no issue.

Output of conda info

``` conda info active environment : base active env location : /Users/mfansler/mambaforge-arm64 shell level : 1 user config file : /Users/mfansler/.condarc populated config files : /Users/mfansler/mambaforge-arm64/.condarc /Users/mfansler/.condarc conda version : 4.11.0 conda-build version : 3.21.8 python version : 3.9.10.final.0 virtual packages : __osx=12.2.1=0 __unix=0=0 __archspec=1=arm64 base environment : /Users/mfansler/mambaforge-arm64 (writable) conda av data dir : /Users/mfansler/mambaforge-arm64/etc/conda conda av metadata url : None channel URLs : https://conda.anaconda.org/conda-forge/osx-arm64 https://conda.anaconda.org/conda-forge/noarch https://conda.anaconda.org/bioconda/osx-arm64 https://conda.anaconda.org/bioconda/noarch https://conda.anaconda.org/merv/osx-arm64 https://conda.anaconda.org/merv/noarch package cache : /Users/mfansler/mambaforge-arm64/pkgs /Users/mfansler/.conda/pkgs envs directories : /Users/mfansler/mambaforge-arm64/envs /Users/mfansler/.conda/envs platform : osx-arm64 user-agent : conda/4.11.0 requests/2.27.1 CPython/3.9.10 Darwin/21.3.0 OSX/12.2.1 UID:GID : 501:20 netrc file : None offline mode : False ```

[github-actions[bot]]

Hi there, thank you for your contribution!

This issue has been automatically marked as stale because it has not had recent activity. It will be closed automatically if no further activity occurs.

If you would like this issue to remain open please:

1. Verify that you can still reproduce the issue at hand
2. Comment that the issue is still reproducible and include:
   • What OS and version you reproduced the issue on
   • What steps you followed to reproduce the issue

NOTE: If this issue was closed prematurely, please leave a comment.

Thanks!

[mfansler]

This is still broken and reproducible. OP has reproduce steps. Only conda info changes are:

   conda version : 4.12.0
virtual packages : __osx=12.3=0

Same conda-build version (3.12.8).

[mfansler]

Why did this get "stale" labeled so quickly? There hasn't even been a new conda-build release since OP.

[xiaoqiangwang]

I saw the same crash of otool on one executable. This is the otool version info.

otool(1): Apple Inc. version cctools-973.0.1
disassmbler: LLVM version 14.0.3

The otool from XCode 13.4 runs just ok.

llvm-otool(1): Apple Inc. version cctools-994.1
otool(1): Apple Inc. version cctools-994.1
disassmbler: LLVM version 13.1.6, (clang-1316.0.21.2.5)

[LunarLanding]

Editing out this line https://github.com/conda/conda-build/blob/f666e4e8af26a71a73ea4c700e1a85e3453724ba/conda_build/os_utils/macho.py#L150 and hard-coding /usr/bin/[tool name] allows conda to use the system installed xcode versions of otool and install_name_tool.

[jaimergp]

You can also do something like this in your build.sh:

if [[ $target_platform == osx-* ]]; then
    for toolname in "otool" "install_name_tool"; do
        tool=$(find "${BUILD_PREFIX}/bin/" -name "*apple*-$toolname")
        mv "${tool}" "${tool}.bak"
        ln -s "/Library/Developer/CommandLineTools/usr/bin/${toolname}" "$tool"
    done
fi