libpython not loaded under Anaconda

Chris00 commented 4 months ago

Bug Description

While correctly detected, libpython*.dylib is not found at runtime under miniconda.

Steps to Reproduce

Backtrace

--- stdout
  cargo:rerun-if-env-changed=PYO3_CROSS
  cargo:rerun-if-env-changed=PYO3_CROSS_LIB_DIR
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_VERSION
  cargo:rerun-if-env-changed=PYO3_CROSS_PYTHON_IMPLEMENTATION
  cargo:rerun-if-env-changed=PYO3_PRINT_CONFIG

  -- PYO3_PRINT_CONFIG=1 is set, printing configuration and halting compile --
  implementation=CPython
  version=3.12
  shared=true
  abi3=false
  lib_name=python3.12
  lib_dir=/Users/runner/miniconda3/lib
  executable=/Users/runner/miniconda3/bin/python
  pointer_width=64
  build_flags=
  suppress_build_script_link_lines=false

  note: unset the PYO3_PRINT_CONFIG environment variable and retry to compile with the above config
...
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 3.78s
     Running `target/debug/pyo3-test`
dyld[6132]: Library not loaded: @rpath/libpython3.12.dylib
  Referenced from: <D47A24F9-530D-36D3-A2DF-66D920E7E1B9> /Users/runner/work/pyo3-test/pyo3-test/target/debug/pyo3-test
  Reason: tried: '/Users/runner/work/pyo3-test/pyo3-test/target/debug/deps/libpython3.12.dylib' (no such file), '/Users/runner/work/pyo3-test/pyo3-test/target/debug/libpython3.12.dylib' (no such file), '/Users/runner/.rustup/toolchains/stable-aarch64-apple-darwin/lib/rustlib/aarch64-apple-darwin/lib/libpython3.12.dylib' (no such file), '/Users/runner/.rustup/toolchains/stable-aarch64-apple-darwin/lib/libpython3.12.dylib' (no such file), '/Users/runner/lib/libpython3.12.dylib' (no such file), '/usr/local/lib/libpython3.12.dylib' (no such file), '/usr/lib/libpython3.12.dylib' (no such file, not in dyld cache)
/Users/runner/work/_temp/23791571-1fd0-4d1a-b32c-b63b783c7030.sh: line 5:  6132 Abort trap: 6           cargo run
Error: Process completed with exit code 134.

Your operating system and version

MacOS (Github runner) — also on Linux

Your Python version (`python --version`)

Python 3.12

Your Rust version (`rustc --version`)

rustc 1.78.0 (9b00956e5 2024-04-29)

Your PyO3 version

0.21.2

How did you install python? Did you use a virtualenv?

Anaconda

Additional Info

No response

davidhewitt commented 4 months ago

Looks like #1554

Chris00 commented 4 months ago

Yes, that other issue looks similar (it lacks a bit of details to tell it is the very same).

Incidentally, making a symbolic link to libpython to target/debug/deps (for debug builds obviously) solves the issue. Note that setting LD_LIBRARY_PATH=$CONDA_PREFIX/lib (or setting DYLD_LIBRARY_PATH on MacOSX) sort of solves the issue too but it may cause strange behaviors because there are many other libraries in $CONDA_PREFIX/lib and you do not want those to be used instead of the system ones.

Since it seems that anaconda will not solve this issue anytime soon, shouldn't PyO3 attempt to work around it?

PyO3 / pyo3