Questions on INCREF/DECREF synchronization

[MichaelSuen-thePointer]

-- This is not an official python repo so I put discussion here. --

I've just read your PEP on biased ref counting. After reading the code, I have a few question on synchronization of inc/dec ref.

Assuming the following situation:

• A non-owning thread calling INCREF, increase the shared refcnt with seq_cst order (thread A atomic_inc(shared_refcnt, 1, seq_cst))
• At the same time, the owning-thread calling DECREF, decrease the local refcnt to 0 (thread B local_refcnt -= 1)
• The owning thread then check the shared refcnt with relaxed atomic load (thread B atomic_load(shared_refcnt, relaxed))

My question is, is it possible that the INCREF to the shared refcnt not visible to the owning-thread yet, causing its checking on shared refcnt returns zero, then erroneously free the object?

[colesbury]

In general, you must already hold a strong reference to an object before you incref it. A notable exception is weak references, which can create a strong reference from a non-owning reference. They require special handling. When we first create a weak reference to an object, we set the _Py_REF_MAYBE_WEAKREF flag on the shared refcount. This disables the fast-path deallocation: if _Py_REF_MAYBE_WEAKREF is set (or _Py_REF_QUEUED or _Py_REF_MERGED), the owning thread goes through the "slow" deallocation path, which involves an atomic cmpxchg on the shared refcount, synchronizing with any thread that is trying to incref.

The dict and list accesses that avoid locking have similar requirements. They set the _Py_REF_MAYBE_WEAKREF flag the first time a non-owning thread attempts to load an object (and that happens under the dict or list lock).

[colesbury]

Here is the slow path for reference:

https://github.com/colesbury/nogil-3.12/blob/cedde4f5ec3759ad723c89d44738776f362df564/Objects/object.c#L2488-L2504

[MichaelSuen-thePointer]

I see, thank you for the explanation.