search

lericson / pylibmc

A Python wrapper around the libmemcached interface from TangentOrg.
http://sendapatch.se/projects/pylibmc/
BSD 3-Clause "New" or "Revised" License
479 stars 137 forks source link

Fatal Python error: Python memory allocator called without holding the GIL #254

Closed jamadden closed 4 years ago

jamadden commented 4 years ago

Using pylibmc in Python 3.7 or later when $PYTHONDEVMODE is set can cause Python to terminate with the error "Fatal Python error: Python memory allocator called without holding the GIL". This happens at least during calls to Client.get_multi.

It appears that PylibMC_Client_get_multi drops the GIL and then calls pylibmc_memcached_fetch_multi which in turn uses PyMem_New, which is documented as requiring the GIL ("Warning: The GIL must be held when using these functions.").

lericson commented 4 years ago

I'll look into this; seems like changing to PyMem_RawMalloc and friends would fix it in the case where memory is being allocated and free'd during GIL-less operation.