pytest-dev / pytest-randomly

:game_die: Pytest plugin to randomly order tests and control random.seed
MIT License
630 stars 30 forks source link

Calls to std::ostringstream.str() in pybind11 modules do not work on Python 3.12 #655

Open taranu opened 1 week ago

taranu commented 1 week ago

Python Version

3.12

pytest Version

8.3.3

Package Version

3.15.0

Description

Making a simple pybind11 module that binds a function calling ostringstream.str() appears to break in pytest with pytest-randomly on py3.12 (but not 3.11). To reproduce:

testrandomly.cc

#include <pybind11/pybind11.h>
#include <pybind11/stl.h>

namespace py = pybind11;

#include <sstream>

std::string to_string_float(float value) {
    std::ostringstream out;
    out << value;
    return out.str();
}

PYBIND11_MODULE(testrandomly, m)
{
    m.def("to_string_float", to_string_float);
}

Compiled with conda-forge provided g++ 14.1.0 and pybind11 2.13.1 by: g++ -Wall -shared -fPIC -std=c++17 $(python3 -m pybind11 --includes) testrandomly.cc -o testrandomly$(python3-config --extension-suffix)

test_randomly.py

from testrandomly import to_string_float

assert to_string_float(1.0) == "1"

pytest --randomly-seed=0 test_randomly.py works on py3.11 with any seed. pytest -p no:randomly test_randomly.py also works on py3.12. pytest --randomly-seed=0 test_randomly.py fails on py3.12 with any seed like so:

test_randomly.py:3: in <module>
    assert to_string_float(1.0) == "1"
E   AssertionError: assert '' == '1'
E    +  where '' = <built-in method to_string_float of PyCapsule object at 0x7f4737916af0>(1.0)

std::to_string works just fine, hence I suspect that this is a strange interaction with ostringstream. It's possible that this is a pybind11 or py3.12 bug but I figured I'd file an issue here first because I can only reproduce it with pytest-randomly enabled and can't guess at why.

adamchainz commented 1 week ago

Uhhh it would be incredibly surprising if anything in pytest-randomly was doing more than incidentally triggering this. My guess would be that it's one of the things that pytest-randomly imports that's actually fiddling with something that the function relies on.

Try editing your local copy to remove the optional imports, like xdist, factory boy, etc.:

https://github.com/pytest-dev/pytest-randomly/blob/01a0f6c39754a41f0fd7b79c22f94ab2815ffe09/src/pytest_randomly/__init__.py#L25-L66

I think it will be a bug in one of those, or indeed pybind11 or Python 3.12.