Python subclass finalizer not called when object is deleted from C++

[torokati44]

Given this snippet:

import cppyy

cppyy.cppdef("""
#include <iostream>
class Base {
public:
    virtual ~Base() { std::cout << "Base destructor" << std::endl; } 
};
""")

class Derived(cppyy.gbl.Base):
    def __del__(self):
        print("Derived finalizer")

o1 = Derived()
del o1

cppyy.cppdef("""
void delet_this(Base *p) {
    delete p;
}
""")

o2 = Derived()
o2.__python_owns__ = False
cppyy.gbl.delet_this(o2)

I would expect the output to be:

Derived finalizer
Base destructor
Derived finalizer
Base destructor

But instead, it's just this:

Derived finalizer
Base destructor
Base destructor

---

Now, I understand that the del statement is special in Python, and a C++ call can not (necessarily?) replicate it's name-erasing powers easily. And even if it could, if there were multiple references to the deleted object (unlike in this simple example), that would still not cause the object to be destroyed.

However, I am destroying the object "by force" from C++, even if there are multiple references to it still. So, regardless of the references to the object, I would expect that whenever the C++ destructor runs, the Python finalizer runs beforehand.

And, I think, the remaining references (well, which is all of them, this is not del after all) should, after deletion, either:

• Magically disappear from the interpreter as if they were del-ed
• Instantly turn into either of these things:
  • None
  • A "tombstone" object that retains some minimal information about the object that it used to represent, but does no longer exist
  • A "poison" reference, that will cause an error/crash/etc whenever it's accessed in any meaningful way that's no longer possible without the object

[wlav]

The derived finalizer is called when o2 goes out of scope. You can try it, by adding a del o2. Python itself does not guarantee that finalizers are run on program exit: this is because of possible circular references.

Removing all references is likewise impossible, because the references could be on a call frame (which could even be custom, see iPython), or in C-land and thus not necessarily programmatically visible from the garbage collector.

To nonify the pointer held, there needs to be a callback from C++ into cppyy's memory regulator. These hooks exist, but require the C++ framework to use them. However, yes, in this specific case, I can generate the callback in the destructor of the dispatcher.

[torokati44]

To nonify the pointer held, there needs to be a callback from C++ into cppyy's memory regulator. These hooks exist, but require the C++ framework to use them. However, yes, in this specific case, I can generate the callback in the destructor of the dispatcher.

Isn't there a common point of indirection somewhere through which any Python reference accesses any given bound/wrapped C++ object? If there was, the "global" "nonification" could be done by the dispatcher destructor there?

[wlav]

In Python, yes; but in C++, no (unless you consistently use std::shared_ptr). This is why only in this special case it can work, b/c the dispatcher destructor itself can call the hook (or for that matter, it can do the nonifying directly by setting the proxy to nullptr: access is always null-checked).

(And if you wonder why the hooks aren't public/documented: they're not legal C++. Just happen to work on all platforms.)

[torokati44]

In Python, yes; but in C++, no ...

Ooh, that's... too bad. :/

[wlav]

Code in repo will nullify the pointer in this specific case, such that accessing the Python proxy post-deletion will result in a ReferenceError rather than segfault or worse.

[wlav]

Released with cppyy 2.4.2 and its dependencies.