Add Py_mod_gil slot to C extension modules

pandas-dev / pandas

Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more

https://pandas.pydata.org

BSD 3-Clause "New" or "Revised" License

42.57k stars 17.56k forks source link

Add Py_mod_gil slot to C extension modules #59135

Open lysnikolaou opened 2 days ago

lysnikolaou commented 2 days ago

x-ref #59057.

The Py_mod_gil slot is needed so that the GIL is not automatically enabled when these C extension modules get imported.

I checked pandas_datetime for thread-safety and it appears to be okay.
pandas_parser is not thread-safe. A discussion is needed on whether we should care about it though, since the c engine to read_csv and friends is already documented as being thread-unsafe and the thread-safe pyarrow alternative is already there.

WillAyd commented 2 days ago

Is this slot only available in 3.13?

pandas_parser is not thread-safe. A discussion is needed on whether we should care about it though,

Do you know what makes it not thread unsafe? I'm surprised given how small of a module it is.

Definitely more of a nice-to-have on the parser though. Understood its probably a very large undertaking to make the CSV reader overall multi-threaded, and most likely not even worth it with pyarrow