python / cpython

The Python programming language
https://www.python.org
Other
62.71k stars 30.06k forks source link

Concurrent reads from `cell` objects (`LOAD_DEREF`) do not scale well #123358

Open colesbury opened 1 month ago

colesbury commented 1 month ago

Concurrent accesses from multiple threads to the same cell object do not scale well in the free-threaded build. There are two problems:

1) LOAD_DEREF calls PyCell_GetRef, which acquires the per-object lock on the cell instance. We'll need to use _Py_TryXGetRef or similar to safely access the value.

2) PyCell_GetRef returns a strong reference and LOAD_DEREF will need a _PyStackRef to handle objects that use deferred reference counting.

Linked PRs

colesbury commented 1 month ago

This came up in the context of https://github.com/wjakob/nanobind/pull/695

corona10 commented 1 week ago

@colesbury Can I take a look at it?