capi-workgroup / decisions

Discussion and voting on specific issues
4 stars 1 forks source link

Add PyType_GetModuleByDef() to the limited C API #20

Closed vstinner closed 3 months ago

vstinner commented 3 months ago

I propose adding PyType_GetModuleByDef() to the limited C API version 3.13. The function was added to Python 3.9, the function is now well tested. We could add it way earlier to the limited C API.

PR: https://github.com/python/cpython/pull/116937


The PyType_GetModuleByDef() function is needed to access the module state in C function which don't get a module as first parameter, but only an instance of a heap type. Heap types should be created with PyType_FromModuleAndSpec(module, spec, bases) to store the module in the type. PyType_GetModuleByDef(type) gives the module and then PyModule_GetState() gives the moulde state.

Examples of functions which don't get a module argument:

Without PyType_GetModuleByDef(), many static types cannot be converted to heap types.

For example, the following stdlib extensions use PyType_GetModuleByDef()

Modules/_asynciomodule.c
Modules/_bz2module.c
Modules/_collectionsmodule.c
Modules/_csv.c
Modules/_decimal/_decimal.c
Modules/_elementtree.c
Modules/_functoolsmodule.c
Modules/_io/_iomodule.h
Modules/_pickle.c
Modules/_queuemodule.c
Modules/_randommodule.c
Modules/_sqlite/module.h
Modules/_ssl.c
Modules/_ssl.h
Modules/_struct.c
Modules/_testmultiphase.c
Modules/_threadmodule.c
Modules/_xxinterpchannelsmodule.c
Modules/_zoneinfo.c
Modules/arraymodule.c
Modules/cjkcodecs/multibytecodec.c
Modules/clinic/_testmultiphase.c.h
Modules/itertoolsmodule.c
Modules/socketmodule.c
Python/Python-tokenize.c

In PyPI top 8,000 projects (at 2024-03-16), I found one project using it: pyzstd (0.15.9). I suppose that other projects still use static types (as Cython), they were not updated to use heap types yet. PyType_FromMetaclass() and PyType_GetModuleByDef() were only added "recently": Python 3.12 and 3.11. I suppose that it's hard to move to heap types without these functions, and support Python up to Python 3.6.

gvanrossum commented 3 months ago

We use this in _asynciomodule.c and it serves its purpose. So +1 from me.

vstinner commented 3 months ago

@zooba: Are you fine with this it? Apparently, the other SC members are fine with it according 👍️ reactions and comments :-)

zooba commented 3 months ago

Yeah, it's fine by me

vstinner commented 3 months ago

The whole working group is in favor of adding the function. Let me add this function in this case. I close the issue.