BUG: pip install darshan on Mac and shared libs

This is reproducible on x86 and Apple silicon Mac laptops with Python 3.11 (from brew) and OS version 13.3.1. I noticed it while helping Yaris get onboarded today.

python3.11 -m venv ~/python_venvs/py_311_debug
source ~/python_venvs/py_311_debug/bin/activate
python -m pip install darshan
python -c "import darshan"

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/__init__.py", line 17, in <module>
    from darshan.report import DarshanReport
  File "/Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/report.py", line 10, in <module>
    import darshan.backend.cffi_backend as backend
  File "/Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/backend/cffi_backend.py", line 49, in <module>
    libdutil = find_utils(ffi, libdutil)
               ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/discover_darshan.py", line 222, in find_utils
    raise RuntimeError('Could not find libdarshan-util.so! Is darshan-util installed? Please ensure one of the the following: 1) export LD_LIBRARY_PATH=<path-to-libdarshan-util.so>, or 2) darshan-parser can found using the PATH variable, or 3) pkg-config can resolve pkg-config --path darshan-util, or 4) install a wheel that includes darshan-utils via pip.')
RuntimeError: Could not find libdarshan-util.so! Is darshan-util installed? Please ensure one of the the following: 1) export LD_LIBRARY_PATH=<path-to-libdarshan-util.so>, or 2) darshan-parser can found using the PATH variable, or 3) pkg-config can resolve pkg-config --path darshan-util, or 4) install a wheel that includes darshan-utils via pip.

The shared lib that gets installed by the wheel is here: /Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/.dylibs/libdarshan-util.0.dylib

If I move the file to remove the .0:

mv /Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/.dylibs/libdarshan-util.0.dylib /Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/.dylibs/libdarshan-util.dylib
and then set DYLD_FALLBACK_LIBRARY_PATH to the containing directory, the problems goes away.

The problem comes back if I restore the original shared lib name from wheel install:

mv /Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/.dylibs/libdarshan-util.dylib /Users/treddy/python_venvs/py_311_debug/lib/python3.11/site-packages/darshan/.dylibs/libdarshan-util.0.dylib

For Applie silicon, I don' t think we have a stable release with wheels available, but Yaris is on x86, so that should work to auto-load the shared lib for darshan-util.

darshan-hpc / darshan

BUG: pip install darshan on Mac and shared libs #930