Failed to run on M1 with anaconda?

[Ucag]

I can successfully build the executable but it failed when run it. Error below occurs:

dyld[53265]: Library not loaded: @rpath/libpython3.12.dylib
  Referenced from: <29BA5D08-624D-3958-993E-175668870FA1> /Users/ucag/practice/call_python_from_rust/target/debug/call_python_from_rust
  Reason: no LC_RPATH's found
[1]    53265 abort      ./target/debug/call_python_from_rust

The code is copied from the doc.

main.rs

use pyo3::prelude::*;
use pyo3::types::IntoPyDict;

fn main() -> PyResult<()> {
    Python::with_gil(|py| {
        let sys = py.import_bound("sys")?;
        let version: String = sys.getattr("version")?.extract()?;

        let locals = [("os", py.import_bound("os")?)].into_py_dict_bound(py);
        let code = "os.getenv('USER') or os.getenv('USERNAME') or 'Unknown'";
        let user: String = py.eval_bound(code, None, Some(&locals))?.extract()?;

        println!("Hello {}, I'm Python {}", user, version);
        Ok(())
    })
}

Cargo.toml

[package]
name = "call_python_from_rust"
version = "0.1.0"
edition = "2021"

[dependencies.pyo3]
version = "0.21.2"
features = ["auto-initialize"]

I can build it without any error reported, but the error appears at runtime. For inspecting the built binary, use command below

otool -L target/debug/call_python_from_rust

target/debug/call_python_from_rust:
        @rpath/libpython3.12.dylib (compatibility version 3.12.0, current version 3.12.0)
        @rpath/libiconv.2.dylib (compatibility version 9.0.0, current version 9.1.0)
        /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1345.100.2)

[Ucag]

After really searching for hours, I finally solve this issue. Maybe this can be a documentation PR? I'm not sure. For helping people meet the same issue. The solution is below:

TL;DR;

Add a build.rs to root of the project aligned to the Cargo.toml file.

use std::process::Command;
pub fn main(){
    let python_inline_script = "from distutils import sysconfig;print(sysconfig.get_config_var('LIBDIR'))";
    let output = Command::new("python")
        .arg("-c")
        .arg(python_inline_script)
        .output()
        .expect("Failed to execute command");
    if !output.status.success() {
        panic!("Python command failed");
    }
    let python_lib_path = String::from_utf8_lossy(&output.stdout);
    println!("cargo:rustc-link-arg=-Wl,-rpath,{}", python_lib_path);
}

Then remove the target directory and run cargo build.

For people who's interested in what's going on here:

The build script just simply add rpath to the linker. From the output of command

otool -L target/debug/call_python_from_rust

we can see that the built binary using @rpath to find used dynamic libraries. Here is the point, rpath differs from LD_LIBRARY_PATH. Value of rpath will be hard coded into binary header. Thus, we need to provide linker the right rpath.

Actually, rpath should be searched in system library path. This problem won't happen if I use python lib shipped with macOS. It just happens with anaconda. I searched a lot find out that many people have the similar issue when use with anaconda.

There are some other ways to tell the system to search for the library, like use DYLD_FALLBACK_LIBRARY_PATH, but it's not easy to use. For details, see issue 1800.

Actually, this still does not totally solve the issue. Because the library path is hard coded, when publish the compiled binary to others, it will be a problem. This is only a just fine way to make things work.