search

PyO3 / pyo3

Rust bindings for the Python interpreter
https://pyo3.rs
Other
11.44k stars 693 forks source link

many 128-bit number conversion tests fail on s390x-unknown-linux-gnu (big-endian) #4290

Closed decathorpe closed 6 days ago

decathorpe commented 6 days ago

I'm currently working on updating the packages for PyO3 on Fedora Linux to the latest version, but new test failures are currently blocking my progress. In turn, it blocks fixing Python packages that require pyo3 v0.22 for Python 3.13 support (like jiter / pydantic).

On s390x-unknown-linux-gnu (the only big-endian architecture we currently support), many tests that handle converting to / from 128-bit integer types (i128 and u128) fail:

It looks like handling of 128-bit numbers changed a fair bit between pyo3 0.21.2 and 0.22.0, notably this commit (linking to the single commit from the PR instead of the squashed changes that ended up in "main" for clarity): https://github.com/PyO3/pyo3/pull/4184/commits/b10d55a9bdd35a49baf565ba41047c9083950dae

Looks like these changes introduced regressions in 128-bit number handling with Python <=3.12 on big-endian architectures, and the new code also don't appear to work correctly with Python 3.13 on big-endian architectures.

In particular, it looks like the #[cfg(Py_3_13)] block here doesn't cover all the code it should: https://github.com/PyO3/pyo3/pull/4184/commits/b10d55a9bdd35a49baf565ba41047c9083950dae#diff-431b01610fad3912c47a930d1447235427b7102b09c1e70fb80d72df74631635L229-R276

... which makes the the return value change from Ok(<$rust_type>::from_le_bytes(buffer)) to Ok(<$rust_type>::from_ne_bytes(buffer)) (note from_LE_bytes vs. from_NE_bytes) for the non-python-3.13 code path too, from what I can tell - which would explain the regression in the tests on Python 3.12.

Test failures from pyo3 v0.22.0 on Python 3.12.3: 

failures:
---- conversions::std::num::test_128bit_integers::test_i128_max stdout ----
thread 'conversions::std::num::test_128bit_integers::test_i128_max' panicked at src/conversions/std/num.rs:497:13:
assertion `left == right` failed
  left: 170141183460469231731687303715884105727
 right: -129
 --- (REDACTED because otherwise GitHub says the ticket body is too long) ---
---- conversions::std::num::tests::u128::to_py_object_and_back stdout ----
thread 'conversions::std::num::tests::u128::to_py_object_and_back' panicked at src/conversions/std/num.rs:709:5:
assertion `left == right` failed
  left: 163495043481544652367168268414482382848
 right: 123
failures:
    conversions::std::num::test_128bit_integers::test_i128_max
    conversions::std::num::test_128bit_integers::test_i128_min
    conversions::std::num::test_128bit_integers::test_i128_roundtrip
    conversions::std::num::test_128bit_integers::test_nonzero_i128_max
    conversions::std::num::test_128bit_integers::test_nonzero_i128_min
    conversions::std::num::test_128bit_integers::test_nonzero_i128_roundtrip
    conversions::std::num::test_128bit_integers::test_nonzero_u128_roundtrip
    conversions::std::num::test_128bit_integers::test_u128_roundtrip
    conversions::std::num::tests::i128::to_py_object_and_back
    conversions::std::num::tests::nonzero_i128::to_py_object_and_back
    conversions::std::num::tests::nonzero_u128::to_py_object_and_back
    conversions::std::num::tests::u128::to_py_object_and_back
test result: FAILED. 663 passed; 12 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s

Test failures from pyo3 v0.21.2 on Python 3.13.0b2: 

failures:
---- conversions::std::num::test_128bit_integers::test_i128_overflow stdout ----
thread 'conversions::std::num::test_128bit_integers::test_i128_overflow' panicked at src/conversions/std/num.rs:482:13:
assertion failed: err.is_instance_of::(py)
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
---- conversions::std::num::test_128bit_integers::test_nonzero_i128_overflow stdout ----
thread 'conversions::std::num::test_128bit_integers::test_nonzero_i128_overflow' panicked at src/conversions/std/num.rs:535:13:
assertion failed: err.is_instance_of::(py)
---- conversions::std::num::test_128bit_integers::test_nonzero_u128_overflow stdout ----
thread 'conversions::std::num::test_128bit_integers::test_nonzero_u128_overflow' panicked at src/conversions/std/num.rs:544:13:
assertion failed: err.is_instance_of::(py)
---- conversions::std::num::test_128bit_integers::test_u128_overflow stdout ----
thread 'conversions::std::num::test_128bit_integers::test_u128_overflow' panicked at src/conversions/std/num.rs:491:13:
assertion failed: err.is_instance_of::(py)
failures:
    conversions::std::num::test_128bit_integers::test_i128_overflow
    conversions::std::num::test_128bit_integers::test_nonzero_i128_overflow
    conversions::std::num::test_128bit_integers::test_nonzero_u128_overflow
    conversions::std::num::test_128bit_integers::test_u128_overflow
test result: FAILED. 629 passed; 4 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.10s
error: test failed, to rerun pass `--lib`