MacOS .dylib libraries have broken `-install_name`

[ndevenish]

The .dylib -install_name/Library ID values appear to be broken for cctbx packages on MacOS in a way that makes it hard to successfully link to from downstream packages e.g.

$ mamba create -yp ENV cctbx-base
...
$ otool -D ENV/lib/libscitbx_boost_python.dylib
ENV/lib/libscitbx_boost_python.dylib:
lib/libscitbx_boost_python.dylib

so, has a relative ID, instead of having an RPATH and being RPATH-relative, like the boost libraries:

$ otool -D ENV/lib/libboost_python310.dylib
ENV/lib/libboost_python310.dylib:
@rpath/libboost_python310.dylib

$ otool -l ENV/lib/libboost_python310.dylib
...
          cmd LC_RPATH
      cmdsize 32
         path @loader_path/ (offset 12)

There appears to be no way to actually override this when linking, because the Mac ecosystem seems to rely on a library accurately knowing it's own ID, leaving the only options to either fiddle with install_name_tool (on the built package, because cannot change the upstream library, because conda), or setting a DYLD_LIBRARY_PATH (might be technically possible on a conda-forge package, but is definitely bad practice).

This should either be an @rpath/libscitbx_boost_python.dylib (so the builder is responsible for adding to the RPATH search folders, or an absolute path (in which case I believe conda-build prefix-rewriting will take care of the issue).

Specific Example

As a specific example, setting up with:

$ mamba create -yp ENV cctbx-base
$ echo "int main(){}" > test.c
$ clang test.c -o test ENV/lib/libscitbx_boost_python.dylib

Running test cannot find the linked library:

$ ./test
dyld: Library not loaded: lib/libscitbx_boost_python.dylib
  Referenced from: /private/var/folders/yq/n9kwymws3q317f02dwpl9jrr0000gn/T/tmp.2K5uCIxY/./test
  Reason: image not found
Abort trap: 6

... unless you run in a CWD that has the library in the relative path:

$ cd ENV
$ ../test

(this fails, because minimal test and boost isn't linked, but fails further down the chain, after it's found libscitbx_boost_python at runtime).

[bkpoon]

Huh, I'll fix it so that the conda-build process fixes that path correctly.

[bkpoon]

This should work with 2022.3 release. Create and activate an environment with that version and then you can compile your test.c with

clang test.c -o test -Wl,-rpath,${CONDA_PREFIX}/lib ${CONDA_PREFIX}/lib/libscitbx_boost_python.dylib ${CONDA_PREFIX}/lib/libpython3.10.dylib

[ndevenish]

Perfect! I can now import dependent modules without setting DYLD_LIBRARY_PATH 😃