PyO3 / rust-numpy

PyO3-based Rust bindings of the NumPy C-API
BSD 2-Clause "Simplified" License
1.11k stars 106 forks source link

Inconsistent `usize` on Windows platforms #404

Closed jeertmans closed 11 months ago

jeertmans commented 11 months ago

Hello, I am writing a small Python library thanks to PyO3 and I use the NumPy bindings you provide to return NumPy arrays.

When testing my code on GitHub workflows, it succeeds on ubuntu-latest and macos-latest, but fails on windows-latest. I guess this is because usize is u32 on the former platforms, and u64 on Windows, but I don't understand why it does not work...

The error message is the following:

`Result::unwrap()` on an `Err` value: PyErr { type: <class 'TypeError'>, value: TypeError('type mismatch:\n from=uint32, to=uint64'), traceback: None }

where the faulty line is unwrapping the .extract() call in:

use pyo3::prelude::*;
use pyo3::{types:IntoPyDict, Python};
use numpy::PyArray2;

Python::with_gil(|py| {
    let np = py.import("numpy").unwrap();
    let locals = [("np", np)].into_py_dict(py);
    let code = "np.empty((0, 0), dtype=np.uint)";

    let expected: &PyArray2<usize> = py
        .eval(code, Some(locals), None)
        .unwrap()
        .extract()
        .unwrap(); // Error here
});

Any idea how to fix this, except by forcing either u32 or u64?

adamreichold commented 11 months ago

Any idea how to fix this, except by forcing either u32 or u64?

In our tests, we force u32 or u64 because NumPy is aligned with C's integer hierarchy and this leads to differences between Windows and Linux: numpy.uint matches C's unsigned long and hence is 64-bits wide on Windows (using LLP64 data model) but 32-bits wide on Linux (using LP64 data model), c.f. https://en.wikipedia.org/wiki/64-bit_computing#64-bit_data_models.

adamreichold commented 11 months ago

(You could use std::ffi::c_ulong instead of usize which should match NumPy's behaviour but of course this does change the semantics.)

jeertmans commented 11 months ago

Oh thanks for the answer @adamreichold, I thought usize would always be the size of a word, regardless of the program, but it makes things clearer now :)

adamreichold commented 11 months ago

So this can be closed or do see something actionable we should do here, e.g. extend the documentation?

jeertmans commented 11 months ago

I think this might be worth adding some words in the documentation, but I can close this issue and create another one if you prefer?