astral-sh / rye

a Hassle-Free Python Experience
https://rye.astral.sh
MIT License
12.08k stars 425 forks source link

rye toolchain not being picked up in maturin/pyo3 build #1158

Open nazq opened 2 weeks ago

nazq commented 2 weeks ago

Steps to Reproduce

Full repro here git@github.com:nazq/maturin_cargo_build_repro.git

I have a pyo3 py/rust project. I've stripped it down to, a simple Rust function, a rust function which links to pyo3, and one which links to pyo3 and exposes itself to the python env(all shown below). cargo test fails with a linker error. Which looks like it can't statically build vs the libpython.so. I have toolchains managed by rye and all are present and contain the dev libs:

🐍 v3.12.2 ❯  find ~/.rye/py -iname "*libpython*.so"
/home/me/.rye/py/cpython@3.8.18/lib/libpython3.8.so
/home/me/.rye/py/cpython@3.8.18/lib/libpython3.so
/home/me/.rye/py/cpython@3.11.8/lib/libpython3.so
/home/me/.rye/py/cpython@3.11.8/lib/libpython3.11.so
/home/me/.rye/py/cpython@3.12.2/lib/libpython3.12.so
/home/me/.rye/py/cpython@3.12.2/lib/libpython3.so
/home/me/.rye/py/cpython@3.10.13/lib/libpython3.10.so
/home/me/.rye/py/cpython@3.10.13/lib/libpython3.so
/home/me/.rye/py/cpython@3.9.18/lib/libpython3.9.so
/home/me/.rye/py/cpython@3.9.18/lib/libpython3.so

cargo test fails with (on a py3.10 toolchain):

= note: /usr/bin/ld: cannot find -lpython3.10: No such file or directory collect2: error: ld returned 1 exit status

I was able to get this to build by doing the following (see https://github.com/nazq/maturin_cargo_build_repro/blob/main/Makefile#L4 for details).

$version=3.10
LD_LIBRARY_PATH=$HOME/.rye/py/cpython@3.8.18/lib:$HOME/.rye/py/cpython@3.9.18/lib:$$HOME/.rye/py/cpython@3.10.13/lib:$HOME/.rye/py/cpython@3.11.8/lib:$HOME/.rye/py/cpython@3.12.2/lib:$LD_LIBRARY_PATH \
RUSTFLAGS="-L $HOME/.rye/py/cpython@3.8.18/lib -L $HOME/.rye/py/cpython@3.9.18/lib -L $HOME/.rye/py/cpython@3.10.13/lib -L $HOME/.rye/py/cpython@3.11.8/lib -L $HOME/.rye/py/cpython@3.12.2/lib" \
PYO3_CROSS_PYTHON_VERSION=$version \
cargo test 

Feels like there is an issue with how the rye environments are exposed to the pyo3/cargo build. Happy to cross post there

# Rust code
fn simple_call() -> String {
    "simple result".to_string()
}

fn find_file(file_path: &Path) -> PyResult<PathBuf> {
    let current_path = file_path.to_path_buf();

    if current_path.is_file() {
        return Ok(current_path);
    } else {
        return Err(PyFileNotFoundError::new_err(format!(
            "File not found: {}",
            current_path.display()
        )));
    }
}

#[pyfunction]
pub fn py_check_file(file_path: &str) -> PyResult<String> {
    let path = Path::new(file_path);
    let result = find_file(path)?;
    Ok(result.display().to_string())
}

#[cfg(test)]
mod tests {
    use super::*;

    #[test]
    fn test_simple_call() {
        let expected = "simple result".to_string();
        let result = simple_call();
        assert_eq!(result, expected);
    }

    #[test]
    fn test_find_cfg_file_in_current_directory() {
        let result = find_file(Path::new("Cargo.toml")).unwrap();
        assert_eq!(result, PathBuf::from("Cargo.toml"));
    }

    #[test]
    fn test_py_check_file() {
        let result = py_check_file("Cargo.toml").unwrap();
        assert_eq!(result, "Cargo.toml");
    }

    #[test]
    fn test_py_check_file_fail() {
        let result = py_check_file("Cargo.toml.NO");
        assert!(result.is_err());
    }
}

Expected Result

cargo test should work as is.

Actual Result

cargo test fails with (on a py3.10 toolchain):

= note: /usr/bin/ld: cannot find -lpython3.10: No such file or directory collect2: error: ld returned 1 exit status

Version Info

maturin_cargo_build_repro🐍 v3.10.13 ❯ rye --version rye 0.31.0 commit: 0.31.0 (72e239b8f 2024-03-22) platform: linux (x86_64) self-python: cpython@3.12.2 symlink support: true uv enabled: true

Stacktrace

No response