search

python / cpython

The Python programming language
https://www.python.org/
Other
61.16k stars 29.52k forks source link

Difficult or impossible to figure out how garbage collector and weak references should interact for user-defined extension types #60399

Open 8726d1eb-a365-45b6-b81d-c75988975e5a opened 11 years ago

8726d1eb-a365-45b6-b81d-c75988975e5a commented 11 years ago
BPO 16195
Nosy @jcea, @pitrou, @scoder, @asvetlov, @phmc

Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

Show more details

GitHub fields: ```python assignee = None closed_at = None created_at = labels = ['type-bug', 'docs'] title = 'Difficult or impossible to figure out how garbage collector and weak references should interact for user-defined extension types' updated_at = user = 'https://bugs.python.org/exarkun' ``` bugs.python.org fields: ```python activity = actor = 'scoder' assignee = 'docs@python' closed = False closed_date = None closer = None components = ['Documentation'] creation = creator = 'exarkun' dependencies = [] files = [] hgrepos = [] issue_num = 16195 keywords = [] message_count = 3.0 messages = ['172639', '172641', '198983'] nosy_count = 7.0 nosy_names = ['jcea', 'exarkun', 'pitrou', 'scoder', 'asvetlov', 'docs@python', 'pconnell'] pr_nums = [] priority = 'normal' resolution = None stage = 'needs patch' status = 'open' superseder = None type = 'behavior' url = 'https://bugs.python.org/issue16195' versions = ['Python 2.7', 'Python 3.2', 'Python 3.3', 'Python 3.4'] ```

8726d1eb-a365-45b6-b81d-c75988975e5a commented 11 years ago

There appears to be very little, if any, documentation about how to handle the list at tp_weaklistoffset for types supporting being weak referenced, particularly with respect to garbage collection.

Who owns the list? Who owns the objects in the list? Should the list be traversed by the extension type's tp_traverse? Should it ever be INCREF'd or DECREF'd? Does tp_dealloc have any responsibilities with respect to it?

From looking at CPython's own source, it appears that tp_traverse should not touch it (Yhg1s confirmed this on #python). Cython, at least, appears to have come to an alternate conclusion though - it generates tp_traverse functions which visit the list (and provoke gc assertions to fail on Python 2.7 in debug mode).

Some documentation about what is correct to do would be excellent, and would make it easier to explain to Cython developers what they're doing wrong (or confirm that they're doing something right, in which case there are some more bugs to file against the CPython stdlib, since they do something different from Cython).

pitrou commented 11 years ago

I think Cython is wrong here. The only thing to do is to call PyObject_ClearWeakRefs() in the deallocator. Everything else is handled by the interpreter.

Agreed improving the documentation would be good.

scoder commented 10 years ago

Just as a quick update here: Cython has since then switched to only using PyObject_ClearWeakRefs() and otherwise leaves the handling of the weakref slot to CPython.