vstinner
commented
1 year ago For me, returning a borrowed reference is the same class of bug than issue #57: implicit resource management, it's unclear until when a resource is valid or not.
For the general case of "returning a pointer", I proposed a generic PyResource API which requires calling PyResource_Release() function when the caller is done with the resource: https://github.com/python/cpython/issues/106592
For the specific case of PyObject* borrowed reference, I think that the fix is easier: just return a PyObject* strong reference instead. The caller is expected to call Py_DECREF() to "release the resource".
gvanrossum
encukou
See also https://github.com/capi-workgroup/problems/issues/5 and https://github.com/capi-workgroup/problems/issues/11
Returning a borrowed reference is fundamentally unsafe.
There are conditions where it can be done safely, but each case requires careful analysis, and is often, if not usually, the case that the analysis is done incorrectly.
For example, https://github.com/capi-workgroup/problems/issues/5#issuecomment-1541599164 suggests that it is fine to return a borrowed reference to a tuple element. However, it is only safe if a reference to the tuple is held on the stack. If the only reference to the tuple is on the heap, then borrowing a reference to an element of the tuple is unsafe, as mutation of a heap object could result in the reference to the tuple, and its element vanishing.
I think it is better to say that returning a borrowed reference is always unsafe, than rely on flawed assumptions about performance and faulty reasoning about lifetimes.