search

python / cpython

The Python programming language
https://www.python.org/
Other
60.98k stars 29.44k forks source link

`asyncio` thread-safety issues in the free-threaded build #120974

Open colesbury opened 2 weeks ago

colesbury commented 2 weeks ago

Bug report

  1. fi_freelist isn't thread-safe (move to pycore_freelist.h and follow that pattern)
  2. enter_task, leave_task, and swap_current_task aren't thread-safe due to shared state->current_tasks and borrowed references.
  3. register_task and unregister_task aren't thread-safe due to shared state->asyncio_tasks linked list

For (2) and (3), we can consider using critical sections to protect the accesses to state->current_tasks and state->asyncio_tasks.

Longer term, moving data to per-loop will probably help with multi-threaded scaling.

kumaraditya303 commented 2 weeks ago

I think that the pure Python fallback isn't thread safe either, FWIW I think there are more thread safety issues than these, asyncio code was written with assumption of GIL.

corona10 commented 2 weeks ago

asyncio code was written with assumption of GIL.

Yeah, but for landing PEP 703, we are trying to change modules, not depending on the GIL, even if we need to re-write some parts of the implementation with a conditional flag. (If the new written part requires some performance overhead)