pybind / pybind11

Seamless operability between C++11 and Python
https://pybind11.readthedocs.io/
Other
15.11k stars 2.05k forks source link

[BUG]: tp_traverse occasionally executed before C++ ctor #4869

Open Alcaro opened 9 months ago

Alcaro commented 9 months ago

Required prerequisites

What version (or hash if on master) of pybind11 are you using?

2.11.1

Problem description

Enabling Py_TPFLAGS_HAVE_GC in custom_type_setup will occasionally call type->tp_traverse before the C++ ctor. Depending on what the C++ object contains, this can be anything from harmless (if the allocation is zero filled and traversing it just visits some nulls), to segfault (if all-zeroes is not a legal state for the object, for example trying to iterate a libstdc++ std::unordered_map), or worse (if the allocation is not zero filled, the worst case is a security hole).

I don't know if this is a bug in pybind11, Python itself (calling _PyObject_GC_TRACK in PyType_GenericAlloc in Objects/typeobject.c seems kinda suspicious to me), or in my code, but if it's the latter, the same bug also exists at https://pybind11.readthedocs.io/en/stable/advanced/classes.html#custom-type-setup .

Reproducible example code

#include <pybind11/pybind11.h>
#include <unordered_set>

namespace py = pybind11;

class funny_class;
std::unordered_set<funny_class*> items_that_exist;
class funny_class {
public:
    funny_class() { items_that_exist.insert(this); }
    ~funny_class() { items_that_exist.erase(this); }
};

PYBIND11_MODULE(example, m) {
    py::class_<funny_class> elem_container(m, "fun",
        py::custom_type_setup([](PyHeapTypeObject* heap_type) {
            PyTypeObject* type = &heap_type->ht_type;
            type->tp_flags |= Py_TPFLAGS_HAVE_GC;
            type->tp_traverse = [](PyObject* self_base, visitproc visit, void* arg) {
                auto& self = py::cast<funny_class&>(py::handle(self_base));
                if (!items_that_exist.count(&self))
                    puts("ERROR: Item was traversed without being constructed.");
                return 0;
            };
            type->tp_clear = [](PyObject* self_base) {
                auto& self = py::cast<funny_class&>(py::handle(self_base));
                if (!items_that_exist.count(&self))
                    puts("ERROR: Item was erased without being constructed.");
                return 0;
            };
        }));
    elem_container.def(py::init<>());
}
import example

a = [(example.fun(), example.fun())[1] for _ in range(10000)]

Is this a regression? Put the last known working version here if it is.

Not a regression