search

thouis / numpy-trac-migration

numpy Trac to github issues migration
2 stars 3 forks source link

PyArray_TakeFrom does not release GIL (Trac #2131) #5927

Open numpy-gitbot opened 11 years ago

numpy-gitbot commented 11 years ago

Original ticket http://projects.scipy.org/numpy/ticket/2131 on 2012-05-11 by atmention:fengy-research, assigned to unknown.

PyArray_TakeFrom does not release the GIL in the lengthy memmove.

It shall probably release the GIL after all buffers are copied, and re-obtain the GIL before raising the exception in 'raise' mode.

numpy-gitbot commented 11 years ago

Attachment added by atmention:fengy-research on 2012-06-12: patch

numpy-gitbot commented 11 years ago

atmention:fengy-research wrote on 2012-06-12

I've added a simple patch that releases the GIL if no MASKNA is set.

The fasttake function in array_types.src.c shall avoid setting PyErr. Because the only error it returns is the out of bound error with a return-value of 1, it is easy to simply catch the return-value and emit the python exception in item_selection.c.

A brief test was done, picking up 200,000,000 elements from a 200,000,000 'f8' array, with mode='wrap'.

Without the patch, it took ~15 seconds with 12 threads(~20 seconds doing it in serial). With the patch it took 2 seconds; faster than yet on the same order with multiprocessing(~4 seconds).