Segmentation Fault in multithreaded numpy calls in NPY_AUXDATA_FREE(auxdata)

[ogrisel]

This might be a bug in numpy that might not be thread-safe when called without the GIL but as I am not sure, here is a report:

Reproducer:

import numpy as np
import threading, queue

def f(*args, **kwargs):
    a = np.zeros(1000)
    return a.max()

q = queue.Queue()

def worker():
    while True:
        item = q.get()
        f(item)
        q.task_done()

for i in range(2):
    threading.Thread(target=worker, daemon=True, name=f"worker-{i}").start()

for item in range(100000):
    q.put(item)

q.join()
print("done")

Most of the time, this code segfaults with the following backtrace for the segfaulting thread:

```gdb (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00007ffff744c717 in PyUFunc_ReduceWrapper (context=context@entry=0x7ffff37a9880, operand=operand@entry=0x763584903f0, out=out@entry=0x0, wheremask=wheremask@entry=0x0, axis_flags=axis_flags@entry=0x7ffff37a9920 "\001\024", reorderable=reorderable@entry=1, keepdims=, identity=, loop=, data=, buffersize=, funcname=, errormask=) at numpy/core/src/umath/reduction.c:390 result = 0x76358490460 skip_first_count = 1 iter = 0x76358440a00 op = {0x0, 0x763584903f0, 0x0} op_dtypes = {0x7ffff76eb5a0 , 0x7ffff76eb5a0 , 0x0} it_flags = op_flags = {51445760, 135397376, 4076500728} auxdata = 0x76357823200 result_axes = {1073741823, 0, 1432964862, 21845, 0, 0, 7, 0, 1, 0, 0, 0, 0, 0, 32, 0, -210070176, 32767, 1481180144, 1891, 1, 0, -148001903, 32767, -144562068, 32767, -210070216, 32767, 1481180144, 1891, 1481180144, 1891} op_axes = {0x7ffff37a93a0, 0x0, 0x0} curr_axis = strided_loop = 0x7ffff757a2a0 flags = 0 needs_api = fixed_strides = {0, 8, 0} #2 0x00007ffff758605d in PyUFunc_Reduce (wheremask=0x0, initial=, keepdims=, signature=0x7ffff37a97a0, axes=0x7ffff37a98a0, naxes=, out=, arr=0x763584903f0, ufunc=) at numpy/core/src/umath/ufunc_object.c:2994 ndim = reorderable = ufunc_name = 0x7ffff761f1f0 "maximum" ufuncimpl = iaxes = axis_flags = "\001\024\000\344\377\177\000\000@\037", '\000' , "fh)\367\377\177\000" errormask = 521 descrs = {0x7ffff76eb5a0 , 0x7ffff76eb5a0 , 0x7ffff76eb5a0 } identity = buffersize = 8192 context = {caller = 0x76356cbb510, method = 0x76357541da0, descriptors = 0x7ffff37a9860} result = iaxes = ndim = reorderable = axis_flags = identity = ufunc_name = buffersize = errormask = descrs = ufuncimpl = context = result = axis = i = #3 PyUFunc_GenericReduction (ufunc=, args=, len_args=, kwnames=, operation=) at numpy/core/src/umath/ufunc_object.c:4155 i = naxes = ndim = axes = {0, 32767, 130000, 11, -469762016, 32767, 8000, 0, -469756880, 32767, 35478272, 211012178, 1436352528, 21845, 8000, 0, 1436521568, 21845, 1433340642, 21845, 1436521568, 21845, 8000, 0, 8000, 0, -469756864, 32767, 1436521568, 21845, -143743520, 32767} full_args = {in = 0x76358420d50, out = 0x0} axes_obj = 0x555555948a80 <_Py_NoneStruct> mp = wheremask = 0x0 ret = 0x0 op = 0x763584903f0 indices = signature = {0x76356d05800, 0x76356d05800, 0x76356d05800} out = 0x0 keepdims = 0 initial = 0x0 out_is_passed_by_position = _reduce_type = {0x7ffff762286c "reduce", 0x7ffff7622873 "accumulate", 0x7ffff762533b "reduceat", 0x0} otype_obj = 0x555555948a80 <_Py_NoneStruct> out_obj = 0x555555948a80 <_Py_NoneStruct> indices_obj = 0x0 keepdims_obj = 0x55555593c080 <_Py_FalseStruct> wheremask_obj = 0x55555593c040 <_Py_TrueStruct> override = 0x0 errval = #4 0x00005555557cb811 in cfunction_vectorcall_FASTCALL_KEYWORDS (func=0x763574fb4d0, args=0x7ffff37a9b88, nargsf=, kwnames=) at Objects/methodobject.c:464 --Type for more, q to quit, c to continue without paging-- tstate = 0x5555559f9460 nargs = 7 meth = result = #5 0x000055555580cdc6 in _PyEval_Fast (ts=0x5555559f9460, initial_acc=..., initial_pc=0x5555558d0bcc "\t") at Python/ceval.c:731 nargs = 8123251307008 func = nargsf = res = 0x76357823200 frame_delta = 8123251307008 opcode_targets_base = {0x0, 0x55555580e54c <_PyEval_Fast+6604>, 0x55555580e4f0 <_PyEval_Fast+6512>, 0x55555580e4cd <_PyEval_Fast+6477>, 0x55555580e494 <_PyEval_Fast+6420>, 0x55555580e42d <_PyEval_Fast+6317>, 0x55555580f525 <_PyEval_Fast+10661>, 0x55555580ddca <_PyEval_Fast+4682>, 0x5555555b5ea2 <_PyEval_Fast.cold>, 0x55555580cd0f <_PyEval_Fast+399>, 0x55555580cd02 <_PyEval_Fast+386>, 0x55555580dcde <_PyEval_Fast+4446>, 0x55555580fa35 <_PyEval_Fast+11957>, 0x55555580f93d <_PyEval_Fast+11709>, 0x55555580fb3a <_PyEval_Fast+12218>, 0x55555580e1ea <_PyEval_Fast+5738>, 0x55555580e243 <_PyEval_Fast+5827>, 0x55555580e2f5 <_PyEval_Fast+6005>, 0x55555580e350 <_PyEval_Fast+6096>, 0x55555580e29c <_PyEval_Fast+5916>, 0x55555580d9f1 <_PyEval_Fast+3697>, 0x55555580d983 <_PyEval_Fast+3587>, 0x55555580d7d7 <_PyEval_Fast+3159>, 0x55555580d842 <_PyEval_Fast+3266>, 0x55555580e376 <_PyEval_Fast+6134>, 0x55555580d76c <_PyEval_Fast+3052>, 0x55555580d66d <_PyEval_Fast+2797>, 0x55555580d918 <_PyEval_Fast+3480>, 0x55555580d8ad <_PyEval_Fast+3373>, 0x55555580da5c <_PyEval_Fast+3804>, 0x55555580dac7 <_PyEval_Fast+3911>, 0x55555580db32 <_PyEval_Fast+4018>, 0x55555580db9d <_PyEval_Fast+4125>, 0x55555580dc08 <_PyEval_Fast+4232>, 0x55555580e125 <_PyEval_Fast+5541>, 0x55555580e181 <_PyEval_Fast+5633>, 0x55555580e0b6 <_PyEval_Fast+5430>, 0x55555580d4be <_PyEval_Fast+2366>, 0x55555580d453 <_PyEval_Fast+2259>, 0x55555580dc73 <_PyEval_Fast+4339>, 0x55555580d312 <_PyEval_Fast+1938>, 0x55555580d37d <_PyEval_Fast+2045>, 0x55555580d602 <_PyEval_Fast+2690>, 0x55555580d0b8 <_PyEval_Fast+1336>, 0x55555580d11f <_PyEval_Fast+1439>, 0x55555580d18a <_PyEval_Fast+1546>, 0x55555580e789 <_PyEval_Fast+7177>, 0x55555580d3e8 <_PyEval_Fast+2152>, 0x55555580d597 <_PyEval_Fast+2583>, 0x55555580d529 <_PyEval_Fast+2473>, 0x55555580e751 <_PyEval_Fast+7121>, 0x55555580ce6d <_PyEval_Fast+749>, 0x55555580f3af <_PyEval_Fast+10287>, 0x55555580cfc4 <_PyEval_Fast+1092>, 0x55555580cea9 <_PyEval_Fast+809>, 0x55555580d1f5 <_PyEval_Fast+1653>, 0x55555580e58e <_PyEval_Fast+6670>, 0x55555580ded0 <_PyEval_Fast+4944>, 0x55555580e3e1 <_PyEval_Fast+6241>, 0x55555580cf54 <_PyEval_Fast+980>, 0x55555580e6ea <_PyEval_Fast+7018>, 0x55555580e615 <_PyEval_Fast+6805>, 0x55555580e679 <_PyEval_Fast+6905>, 0x55555580e5d6 <_PyEval_Fast+6742>, 0x55555580df0a <_PyEval_Fast+5002>, 0x55555580df4e <_PyEval_Fast+5070>, 0x55555580dfbf <_PyEval_Fast+5183>, 0x55555580df84 <_PyEval_Fast+5124>, 0x55555580e016 <_PyEval_Fast+5270>, 0x55555580e06f <_PyEval_Fast+5359>, 0x55555580f613 <_PyEval_Fast+10899>, 0x55555580f62e <_PyEval_Fast+10926>, 0x55555580f5ad <_PyEval_Fast+10797>, 0x55555580f2b8 <_PyEval_Fast+10040>, 0x55555580f31d <_PyEval_Fast+10141>, 0x55555580f87c <_PyEval_Fast+11516>, 0x55555580ef19 <_PyEval_Fast+9113>, 0x55555580f6e6 <_PyEval_Fast+11110>, 0x55555580f720 <_PyEval_Fast+11168>, 0x55555580f3cd <_PyEval_Fast+10317>, 0x55555580f493 <_PyEval_Fast+10515>, 0x55555580f4e9 <_PyEval_Fast+10601>, 0x55555580ef4d <_PyEval_Fast+9165>, 0x55555580f3f7 <_PyEval_Fast+10359>, 0x55555580f452 <_PyEval_Fast+10450>, 0x55555580cc3e <_PyEval_Fast+190>, 0x55555580e7f4 <_PyEval_Fast+7284>, 0x55555580e8c3 <_PyEval_Fast+7491>, 0x55555580d6d8 <_PyEval_Fast+2904>, 0x55555580d720 <_PyEval_Fast+2976>, 0x55555580de90 <_PyEval_Fast+4880>, 0x55555580ea87 <_PyEval_Fast+7943>, 0x55555580eb7c <_PyEval_Fast+8188>, 0x55555580eab8 <_PyEval_Fast+7992>, 0x55555580eb46 <_PyEval_Fast+8134>, 0x55555580ec04 <_PyEval_Fast+8324>, 0x55555580efb0 <_PyEval_Fast+9264>, 0x55555580f01c <_PyEval_Fast+9372>, 0x55555580f069 <_PyEval_Fast+9449>, 0x55555580eedf <_PyEval_Fast+9055>, 0x55555580e853 <_PyEval_Fast+7379>, 0x55555580e91b <_PyEval_Fast+7579>, 0x55555580e9b1 <_PyEval_Fast+7729>, 0x55555580f16a <_PyEval_Fast+9706>, 0x55555580ea59 <_PyEval_Fast+7897>, 0x55555580ee4e <_PyEval_Fast+8910>, 0x55555580f577 <_PyEval_Fast+10743>, 0x55555580f0c8 <_PyEval_Fast+9544>, 0x55555580f13c <_PyEval_Fast+9660>, 0x55555580f102 <_PyEval_Fast+9602>, 0x55555580ed42 <_PyEval_Fast+8642>, 0x55555580ece9 <_PyEval_Fast+8553>, 0x55555580ed9c <_PyEval_Fast+8732>, 0x55555580edf5 <_PyEval_Fast+8821>, 0x55555580ec8b <_PyEval_Fast+8459>, 0x55555580ec32 <_PyEval_Fast+8370>, 0x55555580f287 <_PyEval_Fast+9991>, 0x55555580f257 <_PyEval_Fast+9943>, 0x55555580f399 <_PyEval_Fast+10265>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x5555558119c9 <_PyEval_Fast+20041>, 0x555555811a08 <_PyEval_Fast+20104>, 0x555555811a29 <_PyEval_Fast+20137>, 0x555555811a60 <_PyEval_Fast+20192>, 0x55555581080f <_PyEval_Fast+15503>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x55555581228f <_PyEval_Fast+22287>, 0x5555558122fc <_PyEval_Fast+22396>, 0x555555811f94 <_PyEval_Fast+21524>, 0x555555811f27 <_PyEval_Fast+21415>, 0x5555558123fd <_PyEval_Fast+22653>, 0x555555810700 <_PyEval_Fast+15232>, 0x555555810aa1 <_PyEval_Fast+16161>, 0x5555558107a2 <_PyEval_Fast+15394>, 0x555555812001 <_PyEval_Fast+21633>, 0x5555558120db <_PyEval_Fast+21851>, 0x55555581206e <_PyEval_Fast+21742>, 0x5555558121b5 <_PyEval_Fast+22069>, 0x555555812148 <_PyEval_Fast+21960>, 0x555555812222 <_PyEval_Fast+22178>, 0x5555558124d5 <_PyEval_Fast+22869>, 0x55555581246a <_PyEval_Fast+22762>, 0x555555811c3c <_PyEval_Fast+20668>, 0x555555810e0c <_PyEval_Fast+17036>, 0x555555810e79 <_PyEval_Fast+17145>, 0x5555558118d7 <_PyEval_Fast+19799>, 0x5555558125a4 <_PyEval_Fast+23076>, 0x555555812537 <_PyEval_Fast+22967>, 0x555555810cc2 <_PyEval_Fast+16706>, 0x555555810c55 <_PyEval_Fast+16597>, 0x555555810be8 <_PyEval_Fast+16488>, 0x555555810b7b <_PyEval_Fast+16379>, 0x555555810b0e <_PyEval_Fast+16270>, 0x555555811dda <_PyEval_Fast+21082>, 0x555555810d2f <_PyEval_Fast+16815>, 0x555555810d9c <_PyEval_Fast+16924>, 0x555555811aec <_PyEval_Fast+20332>, 0x555555812611 <_PyEval_Fast+23185>, 0x555555810ee6 <_PyEval_Fast+17254>, 0x5555558117d0 <_PyEval_Fast+19536>, 0x55555581062f <_PyEval_Fast+15023>, 0x555555811711 <_PyEval_Fast+19345>, 0x555555811982 <_PyEval_Fast+19970>, 0x55555581282e <_PyEval_Fast+23726>, 0x555555811aac <_PyEval_Fast+20268>, 0x5555558116a2 <_PyEval_Fast+19234>, 0x555555811562 <_PyEval_Fast+18914>, 0x55555581163f <_PyEval_Fast+19135>, 0x5555558115ca <_PyEval_Fast+19018>, 0x555555811944 <_PyEval_Fast+19908>, 0x5555558127eb <_PyEval_Fast+23659>, 0x55555581076d <_PyEval_Fast+15341>, 0x555555811d4a <_PyEval_Fast+20938>, 0x555555811da0 <_PyEval_Fast+21024>, 0x555555811cf2 <_PyEval_Fast+20850>, 0x555555811cac <_PyEval_Fast+20780>, 0x555555811481 <_PyEval_Fast+18689>, 0x55555581236c <_PyEval_Fast+22508>, 0x5555558114dc <_PyEval_Fast+18780>, 0x555555812787 <_PyEval_Fast+23559>, 0x555555812743 <_PyEval_Fast+23491>, 0x5555555b5ea2 <_PyEval_Fast.cold>...} pc = 0x5555558d0bcc "\t" opcode = acc = regs = constants = 0x763574ce9f8 tid = #6 0x00005555556a86bf in _PyEval_Eval (pc=, acc=..., tstate=0x5555559f9460) at Python/ceval_meta.c:2788 ret = cargs = {0x1, 0x763584903f0, 0x555555948a80 <_Py_NoneStruct>, 0x555555948a80 <_Py_NoneStruct>, 0x555555948a80 <_Py_NoneStruct>, 0x55555593c080 <_Py_FalseStruct>, 0x76356eb63d0, 0x55555593c040 <_Py_TrueStruct>, 0x1} prevargs = 0x7ffff37a9d08 ret = cargs = prevargs = #7 _PyFunction_Vectorcall (func=0x763575e7130, stack=0x76358420e30, nargsf=, kwnames=) at Python/ceval_meta.c:3185 tstate = 0x5555559f9460 nargs = 1 nkwargs = ret = 0x0 acc = {as_int64 = 1} err = 0 extra = #8 0x00007ffff739ce87 in forward_ndarray_method (self=, args=0x55555594b2d0 <_Py_EmptyTupleStruct+16>, kwds=0x0, forwarding_callable=0x763575e7130) at numpy/core/src/multiarray/methods.c:89 sargs = 0x76358420e10 ret = i = n = #9 0x00005555557bdbfb in method_vectorcall_VARARGS_KEYWORDS (func=0x7635743c6d0, args=0x7ffff37a9d08, nargsf=, kwnames=) at Objects/descrobject.c:349 tstate = 0x5555559f9460 nargs = 1 argstuple = 0x55555594b2d0 <_Py_EmptyTupleStruct+16> result = 0x0 kwdict = 0x0 meth = #10 0x000055555580cdc6 in _PyEval_Fast (ts=0x5555559f9460, initial_acc=..., initial_pc=0x5555558cec6f "\t") at Python/ceval.c:731 --Type for more, q to quit, c to continue without paging-- nargs = 8123251307008 func = nargsf = res = 0x76357823200 frame_delta = 8123251307008 opcode_targets_base = {0x0, 0x55555580e54c <_PyEval_Fast+6604>, 0x55555580e4f0 <_PyEval_Fast+6512>, 0x55555580e4cd <_PyEval_Fast+6477>, 0x55555580e494 <_PyEval_Fast+6420>, 0x55555580e42d <_PyEval_Fast+6317>, 0x55555580f525 <_PyEval_Fast+10661>, 0x55555580ddca <_PyEval_Fast+4682>, 0x5555555b5ea2 <_PyEval_Fast.cold>, 0x55555580cd0f <_PyEval_Fast+399>, 0x55555580cd02 <_PyEval_Fast+386>, 0x55555580dcde <_PyEval_Fast+4446>, 0x55555580fa35 <_PyEval_Fast+11957>, 0x55555580f93d <_PyEval_Fast+11709>, 0x55555580fb3a <_PyEval_Fast+12218>, 0x55555580e1ea <_PyEval_Fast+5738>, 0x55555580e243 <_PyEval_Fast+5827>, 0x55555580e2f5 <_PyEval_Fast+6005>, 0x55555580e350 <_PyEval_Fast+6096>, 0x55555580e29c <_PyEval_Fast+5916>, 0x55555580d9f1 <_PyEval_Fast+3697>, 0x55555580d983 <_PyEval_Fast+3587>, 0x55555580d7d7 <_PyEval_Fast+3159>, 0x55555580d842 <_PyEval_Fast+3266>, 0x55555580e376 <_PyEval_Fast+6134>, 0x55555580d76c <_PyEval_Fast+3052>, 0x55555580d66d <_PyEval_Fast+2797>, 0x55555580d918 <_PyEval_Fast+3480>, 0x55555580d8ad <_PyEval_Fast+3373>, 0x55555580da5c <_PyEval_Fast+3804>, 0x55555580dac7 <_PyEval_Fast+3911>, 0x55555580db32 <_PyEval_Fast+4018>, 0x55555580db9d <_PyEval_Fast+4125>, 0x55555580dc08 <_PyEval_Fast+4232>, 0x55555580e125 <_PyEval_Fast+5541>, 0x55555580e181 <_PyEval_Fast+5633>, 0x55555580e0b6 <_PyEval_Fast+5430>, 0x55555580d4be <_PyEval_Fast+2366>, 0x55555580d453 <_PyEval_Fast+2259>, 0x55555580dc73 <_PyEval_Fast+4339>, 0x55555580d312 <_PyEval_Fast+1938>, 0x55555580d37d <_PyEval_Fast+2045>, 0x55555580d602 <_PyEval_Fast+2690>, 0x55555580d0b8 <_PyEval_Fast+1336>, 0x55555580d11f <_PyEval_Fast+1439>, 0x55555580d18a <_PyEval_Fast+1546>, 0x55555580e789 <_PyEval_Fast+7177>, 0x55555580d3e8 <_PyEval_Fast+2152>, 0x55555580d597 <_PyEval_Fast+2583>, 0x55555580d529 <_PyEval_Fast+2473>, 0x55555580e751 <_PyEval_Fast+7121>, 0x55555580ce6d <_PyEval_Fast+749>, 0x55555580f3af <_PyEval_Fast+10287>, 0x55555580cfc4 <_PyEval_Fast+1092>, 0x55555580cea9 <_PyEval_Fast+809>, 0x55555580d1f5 <_PyEval_Fast+1653>, 0x55555580e58e <_PyEval_Fast+6670>, 0x55555580ded0 <_PyEval_Fast+4944>, 0x55555580e3e1 <_PyEval_Fast+6241>, 0x55555580cf54 <_PyEval_Fast+980>, 0x55555580e6ea <_PyEval_Fast+7018>, 0x55555580e615 <_PyEval_Fast+6805>, 0x55555580e679 <_PyEval_Fast+6905>, 0x55555580e5d6 <_PyEval_Fast+6742>, 0x55555580df0a <_PyEval_Fast+5002>, 0x55555580df4e <_PyEval_Fast+5070>, 0x55555580dfbf <_PyEval_Fast+5183>, 0x55555580df84 <_PyEval_Fast+5124>, 0x55555580e016 <_PyEval_Fast+5270>, 0x55555580e06f <_PyEval_Fast+5359>, 0x55555580f613 <_PyEval_Fast+10899>, 0x55555580f62e <_PyEval_Fast+10926>, 0x55555580f5ad <_PyEval_Fast+10797>, 0x55555580f2b8 <_PyEval_Fast+10040>, 0x55555580f31d <_PyEval_Fast+10141>, 0x55555580f87c <_PyEval_Fast+11516>, 0x55555580ef19 <_PyEval_Fast+9113>, 0x55555580f6e6 <_PyEval_Fast+11110>, 0x55555580f720 <_PyEval_Fast+11168>, 0x55555580f3cd <_PyEval_Fast+10317>, 0x55555580f493 <_PyEval_Fast+10515>, 0x55555580f4e9 <_PyEval_Fast+10601>, 0x55555580ef4d <_PyEval_Fast+9165>, 0x55555580f3f7 <_PyEval_Fast+10359>, 0x55555580f452 <_PyEval_Fast+10450>, 0x55555580cc3e <_PyEval_Fast+190>, 0x55555580e7f4 <_PyEval_Fast+7284>, 0x55555580e8c3 <_PyEval_Fast+7491>, 0x55555580d6d8 <_PyEval_Fast+2904>, 0x55555580d720 <_PyEval_Fast+2976>, 0x55555580de90 <_PyEval_Fast+4880>, 0x55555580ea87 <_PyEval_Fast+7943>, 0x55555580eb7c <_PyEval_Fast+8188>, 0x55555580eab8 <_PyEval_Fast+7992>, 0x55555580eb46 <_PyEval_Fast+8134>, 0x55555580ec04 <_PyEval_Fast+8324>, 0x55555580efb0 <_PyEval_Fast+9264>, 0x55555580f01c <_PyEval_Fast+9372>, 0x55555580f069 <_PyEval_Fast+9449>, 0x55555580eedf <_PyEval_Fast+9055>, 0x55555580e853 <_PyEval_Fast+7379>, 0x55555580e91b <_PyEval_Fast+7579>, 0x55555580e9b1 <_PyEval_Fast+7729>, 0x55555580f16a <_PyEval_Fast+9706>, 0x55555580ea59 <_PyEval_Fast+7897>, 0x55555580ee4e <_PyEval_Fast+8910>, 0x55555580f577 <_PyEval_Fast+10743>, 0x55555580f0c8 <_PyEval_Fast+9544>, 0x55555580f13c <_PyEval_Fast+9660>, 0x55555580f102 <_PyEval_Fast+9602>, 0x55555580ed42 <_PyEval_Fast+8642>, 0x55555580ece9 <_PyEval_Fast+8553>, 0x55555580ed9c <_PyEval_Fast+8732>, 0x55555580edf5 <_PyEval_Fast+8821>, 0x55555580ec8b <_PyEval_Fast+8459>, 0x55555580ec32 <_PyEval_Fast+8370>, 0x55555580f287 <_PyEval_Fast+9991>, 0x55555580f257 <_PyEval_Fast+9943>, 0x55555580f399 <_PyEval_Fast+10265>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x5555558119c9 <_PyEval_Fast+20041>, 0x555555811a08 <_PyEval_Fast+20104>, 0x555555811a29 <_PyEval_Fast+20137>, 0x555555811a60 <_PyEval_Fast+20192>, 0x55555581080f <_PyEval_Fast+15503>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x55555581228f <_PyEval_Fast+22287>, 0x5555558122fc <_PyEval_Fast+22396>, 0x555555811f94 <_PyEval_Fast+21524>, 0x555555811f27 <_PyEval_Fast+21415>, 0x5555558123fd <_PyEval_Fast+22653>, 0x555555810700 <_PyEval_Fast+15232>, 0x555555810aa1 <_PyEval_Fast+16161>, 0x5555558107a2 <_PyEval_Fast+15394>, 0x555555812001 <_PyEval_Fast+21633>, 0x5555558120db <_PyEval_Fast+21851>, 0x55555581206e <_PyEval_Fast+21742>, 0x5555558121b5 <_PyEval_Fast+22069>, 0x555555812148 <_PyEval_Fast+21960>, 0x555555812222 <_PyEval_Fast+22178>, 0x5555558124d5 <_PyEval_Fast+22869>, 0x55555581246a <_PyEval_Fast+22762>, 0x555555811c3c <_PyEval_Fast+20668>, 0x555555810e0c <_PyEval_Fast+17036>, 0x555555810e79 <_PyEval_Fast+17145>, 0x5555558118d7 <_PyEval_Fast+19799>, 0x5555558125a4 <_PyEval_Fast+23076>, 0x555555812537 <_PyEval_Fast+22967>, 0x555555810cc2 <_PyEval_Fast+16706>, 0x555555810c55 <_PyEval_Fast+16597>, 0x555555810be8 <_PyEval_Fast+16488>, 0x555555810b7b <_PyEval_Fast+16379>, 0x555555810b0e <_PyEval_Fast+16270>, 0x555555811dda <_PyEval_Fast+21082>, 0x555555810d2f <_PyEval_Fast+16815>, 0x555555810d9c <_PyEval_Fast+16924>, 0x555555811aec <_PyEval_Fast+20332>, 0x555555812611 <_PyEval_Fast+23185>, 0x555555810ee6 <_PyEval_Fast+17254>, 0x5555558117d0 <_PyEval_Fast+19536>, 0x55555581062f <_PyEval_Fast+15023>, 0x555555811711 <_PyEval_Fast+19345>, 0x555555811982 <_PyEval_Fast+19970>, 0x55555581282e <_PyEval_Fast+23726>, 0x555555811aac <_PyEval_Fast+20268>, 0x5555558116a2 <_PyEval_Fast+19234>, 0x555555811562 <_PyEval_Fast+18914>, 0x55555581163f <_PyEval_Fast+19135>, 0x5555558115ca <_PyEval_Fast+19018>, 0x555555811944 <_PyEval_Fast+19908>, 0x5555558127eb <_PyEval_Fast+23659>, 0x55555581076d <_PyEval_Fast+15341>, 0x555555811d4a <_PyEval_Fast+20938>, 0x555555811da0 <_PyEval_Fast+21024>, 0x555555811cf2 <_PyEval_Fast+20850>, 0x555555811cac <_PyEval_Fast+20780>, 0x555555811481 <_PyEval_Fast+18689>, 0x55555581236c <_PyEval_Fast+22508>, 0x5555558114dc <_PyEval_Fast+18780>, 0x555555812787 <_PyEval_Fast+23559>, 0x555555812743 <_PyEval_Fast+23491>, 0x5555555b5ea2 <_PyEval_Fast.cold>...} pc = 0x5555558cec6f "\t" opcode = acc = regs = constants = 0x76356ea2b40 tid = #11 0x00005555556a86bf in _PyEval_Eval (pc=, acc=..., tstate=0x5555559f9460) at Python/ceval_meta.c:2788 ret = cargs = {0x0, 0x763584903f0, 0x55555593c080 <_Py_FalseStruct>, 0x7ffff2fa9ac0, 0x7ffff2fa9b60, 0x4, 0x2, 0x3200000008, 0x0} prevargs = 0x0 ret = cargs = prevargs = #12 _PyFunction_Vectorcall (func=0x76356df0470, stack=0x7ffff37a9da8, nargsf=, kwnames=) at Python/ceval_meta.c:3185 tstate = 0x5555559f9460 nargs = 1 nkwargs = ret = 0x0 acc = {as_int64 = 1} err = 0 extra = #13 0x00005555557b3ab4 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7ffff37a9da8, callable=0x76356df0470, tstate=0x5555559f9460) at ./Include/cpython/abstract.h:118 func = res = nargs = #14 method_vectorcall (method=, args=0x55555594b2f0 <_Py_EmptyTupleStruct+48>, nargsf=, kwnames=0x0) at Objects/classobject.c:62 nkwargs = totalargs = newargs_stack = {0x7fffffffd35f, 0x5555559f9460, 0x5555559f9460, 0xc93ca52021d5b00, 0x55555599bc60 <_PyRuntime+640>} newargs = tstate = 0x5555559f9460 self = 0x763568f8e90 func = 0x76356df0470 nargs = 0 result = #15 0x0000555555771a41 in t_bootstrap (boot_raw=0x7635795ba30) at ./Modules/_threadmodule.c:1289 boot = 0x7635795ba30 tstate = 0x5555559f9460 res = --Type for more, q to quit, c to continue without paging-- #16 0x00005555557064bb in pythread_wrapper (arg=) at Python/thread_pthread.h:245 callback = func = 0x5555557719f0 func_arg = 0x7635795ba30 #17 0x00007ffff7d38947 in start_thread (arg=) at pthread_create.c:435 ret = pd = out = unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737278289472, -7107990009691300924, 140737488343902, 140737488343903, 0, 140737269899264, 7108015344611171268, 7108007434102184900}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = #18 0x00007ffff7dc8a44 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100 No locals. (gdb) bt full #0 0x0000000000000000 in ?? () No symbol table info available. #1 0x00007ffff744c717 in PyUFunc_ReduceWrapper (context=context@entry=0x7ffff37a9880, operand=operand@entry=0x763584903f0, out=out@entry=0x0, wheremask=wheremask@entry=0x0, axis_flags=axis_flags@entry=0x7ffff37a9920 "\001\024", reorderable=reorderable@entry=1, keepdims=, identity=, loop=, data=, buffersize=, funcname=, errormask=) at numpy/core/src/umath/reduction.c:390 result = 0x76358490460 skip_first_count = 1 iter = 0x76358440a00 op = {0x0, 0x763584903f0, 0x0} op_dtypes = {0x7ffff76eb5a0 , 0x7ffff76eb5a0 , 0x0} it_flags = op_flags = {51445760, 135397376, 4076500728} auxdata = 0x76357823200 result_axes = {1073741823, 0, 1432964862, 21845, 0, 0, 7, 0, 1, 0, 0, 0, 0, 0, 32, 0, -210070176, 32767, 1481180144, 1891, 1, 0, -148001903, 32767, -144562068, 32767, -210070216, 32767, 1481180144, 1891, 1481180144, 1891} op_axes = {0x7ffff37a93a0, 0x0, 0x0} curr_axis = strided_loop = 0x7ffff757a2a0 flags = 0 needs_api = fixed_strides = {0, 8, 0} #2 0x00007ffff758605d in PyUFunc_Reduce (wheremask=0x0, initial=, keepdims=, signature=0x7ffff37a97a0, axes=0x7ffff37a98a0, naxes=, out=, arr=0x763584903f0, ufunc=) at numpy/core/src/umath/ufunc_object.c:2994 ndim = reorderable = ufunc_name = 0x7ffff761f1f0 "maximum" ufuncimpl = iaxes = axis_flags = "\001\024\000\344\377\177\000\000@\037", '\000' , "fh)\367\377\177\000" errormask = 521 descrs = {0x7ffff76eb5a0 , 0x7ffff76eb5a0 , 0x7ffff76eb5a0 } identity = buffersize = 8192 context = {caller = 0x76356cbb510, method = 0x76357541da0, descriptors = 0x7ffff37a9860} result = iaxes = ndim = reorderable = axis_flags = identity = ufunc_name = buffersize = errormask = descrs = ufuncimpl = context = result = axis = i = #3 PyUFunc_GenericReduction (ufunc=, args=, len_args=, kwnames=, operation=) at numpy/core/src/umath/ufunc_object.c:4155 i = naxes = ndim = axes = {0, 32767, 130000, 11, -469762016, 32767, 8000, 0, -469756880, 32767, 35478272, 211012178, 1436352528, 21845, 8000, 0, 1436521568, 21845, 1433340642, 21845, 1436521568, 21845, 8000, 0, 8000, 0, -469756864, 32767, 1436521568, 21845, -143743520, 32767} full_args = {in = 0x76358420d50, out = 0x0} axes_obj = 0x555555948a80 <_Py_NoneStruct> mp = wheremask = 0x0 ret = 0x0 op = 0x763584903f0 indices = signature = {0x76356d05800, 0x76356d05800, 0x76356d05800} out = 0x0 keepdims = 0 initial = 0x0 out_is_passed_by_position = _reduce_type = {0x7ffff762286c "reduce", 0x7ffff7622873 "accumulate", 0x7ffff762533b "reduceat", 0x0} otype_obj = 0x555555948a80 <_Py_NoneStruct> out_obj = 0x555555948a80 <_Py_NoneStruct> indices_obj = 0x0 keepdims_obj = 0x55555593c080 <_Py_FalseStruct> wheremask_obj = 0x55555593c040 <_Py_TrueStruct> override = 0x0 errval = #4 0x00005555557cb811 in cfunction_vectorcall_FASTCALL_KEYWORDS (func=0x763574fb4d0, args=0x7ffff37a9b88, nargsf=, kwnames=) at Objects/methodobject.c:464 --Type for more, q to quit, c to continue without paging-- tstate = 0x5555559f9460 nargs = 7 meth = result = #5 0x000055555580cdc6 in _PyEval_Fast (ts=0x5555559f9460, initial_acc=..., initial_pc=0x5555558d0bcc "\t") at Python/ceval.c:731 nargs = 8123251307008 func = nargsf = res = 0x76357823200 frame_delta = 8123251307008 opcode_targets_base = {0x0, 0x55555580e54c <_PyEval_Fast+6604>, 0x55555580e4f0 <_PyEval_Fast+6512>, 0x55555580e4cd <_PyEval_Fast+6477>, 0x55555580e494 <_PyEval_Fast+6420>, 0x55555580e42d <_PyEval_Fast+6317>, 0x55555580f525 <_PyEval_Fast+10661>, 0x55555580ddca <_PyEval_Fast+4682>, 0x5555555b5ea2 <_PyEval_Fast.cold>, 0x55555580cd0f <_PyEval_Fast+399>, 0x55555580cd02 <_PyEval_Fast+386>, 0x55555580dcde <_PyEval_Fast+4446>, 0x55555580fa35 <_PyEval_Fast+11957>, 0x55555580f93d <_PyEval_Fast+11709>, 0x55555580fb3a <_PyEval_Fast+12218>, 0x55555580e1ea <_PyEval_Fast+5738>, 0x55555580e243 <_PyEval_Fast+5827>, 0x55555580e2f5 <_PyEval_Fast+6005>, 0x55555580e350 <_PyEval_Fast+6096>, 0x55555580e29c <_PyEval_Fast+5916>, 0x55555580d9f1 <_PyEval_Fast+3697>, 0x55555580d983 <_PyEval_Fast+3587>, 0x55555580d7d7 <_PyEval_Fast+3159>, 0x55555580d842 <_PyEval_Fast+3266>, 0x55555580e376 <_PyEval_Fast+6134>, 0x55555580d76c <_PyEval_Fast+3052>, 0x55555580d66d <_PyEval_Fast+2797>, 0x55555580d918 <_PyEval_Fast+3480>, 0x55555580d8ad <_PyEval_Fast+3373>, 0x55555580da5c <_PyEval_Fast+3804>, 0x55555580dac7 <_PyEval_Fast+3911>, 0x55555580db32 <_PyEval_Fast+4018>, 0x55555580db9d <_PyEval_Fast+4125>, 0x55555580dc08 <_PyEval_Fast+4232>, 0x55555580e125 <_PyEval_Fast+5541>, 0x55555580e181 <_PyEval_Fast+5633>, 0x55555580e0b6 <_PyEval_Fast+5430>, 0x55555580d4be <_PyEval_Fast+2366>, 0x55555580d453 <_PyEval_Fast+2259>, 0x55555580dc73 <_PyEval_Fast+4339>, 0x55555580d312 <_PyEval_Fast+1938>, 0x55555580d37d <_PyEval_Fast+2045>, 0x55555580d602 <_PyEval_Fast+2690>, 0x55555580d0b8 <_PyEval_Fast+1336>, 0x55555580d11f <_PyEval_Fast+1439>, 0x55555580d18a <_PyEval_Fast+1546>, 0x55555580e789 <_PyEval_Fast+7177>, 0x55555580d3e8 <_PyEval_Fast+2152>, 0x55555580d597 <_PyEval_Fast+2583>, 0x55555580d529 <_PyEval_Fast+2473>, 0x55555580e751 <_PyEval_Fast+7121>, 0x55555580ce6d <_PyEval_Fast+749>, 0x55555580f3af <_PyEval_Fast+10287>, 0x55555580cfc4 <_PyEval_Fast+1092>, 0x55555580cea9 <_PyEval_Fast+809>, 0x55555580d1f5 <_PyEval_Fast+1653>, 0x55555580e58e <_PyEval_Fast+6670>, 0x55555580ded0 <_PyEval_Fast+4944>, 0x55555580e3e1 <_PyEval_Fast+6241>, 0x55555580cf54 <_PyEval_Fast+980>, 0x55555580e6ea <_PyEval_Fast+7018>, 0x55555580e615 <_PyEval_Fast+6805>, 0x55555580e679 <_PyEval_Fast+6905>, 0x55555580e5d6 <_PyEval_Fast+6742>, 0x55555580df0a <_PyEval_Fast+5002>, 0x55555580df4e <_PyEval_Fast+5070>, 0x55555580dfbf <_PyEval_Fast+5183>, 0x55555580df84 <_PyEval_Fast+5124>, 0x55555580e016 <_PyEval_Fast+5270>, 0x55555580e06f <_PyEval_Fast+5359>, 0x55555580f613 <_PyEval_Fast+10899>, 0x55555580f62e <_PyEval_Fast+10926>, 0x55555580f5ad <_PyEval_Fast+10797>, 0x55555580f2b8 <_PyEval_Fast+10040>, 0x55555580f31d <_PyEval_Fast+10141>, 0x55555580f87c <_PyEval_Fast+11516>, 0x55555580ef19 <_PyEval_Fast+9113>, 0x55555580f6e6 <_PyEval_Fast+11110>, 0x55555580f720 <_PyEval_Fast+11168>, 0x55555580f3cd <_PyEval_Fast+10317>, 0x55555580f493 <_PyEval_Fast+10515>, 0x55555580f4e9 <_PyEval_Fast+10601>, 0x55555580ef4d <_PyEval_Fast+9165>, 0x55555580f3f7 <_PyEval_Fast+10359>, 0x55555580f452 <_PyEval_Fast+10450>, 0x55555580cc3e <_PyEval_Fast+190>, 0x55555580e7f4 <_PyEval_Fast+7284>, 0x55555580e8c3 <_PyEval_Fast+7491>, 0x55555580d6d8 <_PyEval_Fast+2904>, 0x55555580d720 <_PyEval_Fast+2976>, 0x55555580de90 <_PyEval_Fast+4880>, 0x55555580ea87 <_PyEval_Fast+7943>, 0x55555580eb7c <_PyEval_Fast+8188>, 0x55555580eab8 <_PyEval_Fast+7992>, 0x55555580eb46 <_PyEval_Fast+8134>, 0x55555580ec04 <_PyEval_Fast+8324>, 0x55555580efb0 <_PyEval_Fast+9264>, 0x55555580f01c <_PyEval_Fast+9372>, 0x55555580f069 <_PyEval_Fast+9449>, 0x55555580eedf <_PyEval_Fast+9055>, 0x55555580e853 <_PyEval_Fast+7379>, 0x55555580e91b <_PyEval_Fast+7579>, 0x55555580e9b1 <_PyEval_Fast+7729>, 0x55555580f16a <_PyEval_Fast+9706>, 0x55555580ea59 <_PyEval_Fast+7897>, 0x55555580ee4e <_PyEval_Fast+8910>, 0x55555580f577 <_PyEval_Fast+10743>, 0x55555580f0c8 <_PyEval_Fast+9544>, 0x55555580f13c <_PyEval_Fast+9660>, 0x55555580f102 <_PyEval_Fast+9602>, 0x55555580ed42 <_PyEval_Fast+8642>, 0x55555580ece9 <_PyEval_Fast+8553>, 0x55555580ed9c <_PyEval_Fast+8732>, 0x55555580edf5 <_PyEval_Fast+8821>, 0x55555580ec8b <_PyEval_Fast+8459>, 0x55555580ec32 <_PyEval_Fast+8370>, 0x55555580f287 <_PyEval_Fast+9991>, 0x55555580f257 <_PyEval_Fast+9943>, 0x55555580f399 <_PyEval_Fast+10265>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x5555558119c9 <_PyEval_Fast+20041>, 0x555555811a08 <_PyEval_Fast+20104>, 0x555555811a29 <_PyEval_Fast+20137>, 0x555555811a60 <_PyEval_Fast+20192>, 0x55555581080f <_PyEval_Fast+15503>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x55555581228f <_PyEval_Fast+22287>, 0x5555558122fc <_PyEval_Fast+22396>, 0x555555811f94 <_PyEval_Fast+21524>, 0x555555811f27 <_PyEval_Fast+21415>, 0x5555558123fd <_PyEval_Fast+22653>, 0x555555810700 <_PyEval_Fast+15232>, 0x555555810aa1 <_PyEval_Fast+16161>, 0x5555558107a2 <_PyEval_Fast+15394>, 0x555555812001 <_PyEval_Fast+21633>, 0x5555558120db <_PyEval_Fast+21851>, 0x55555581206e <_PyEval_Fast+21742>, 0x5555558121b5 <_PyEval_Fast+22069>, 0x555555812148 <_PyEval_Fast+21960>, 0x555555812222 <_PyEval_Fast+22178>, 0x5555558124d5 <_PyEval_Fast+22869>, 0x55555581246a <_PyEval_Fast+22762>, 0x555555811c3c <_PyEval_Fast+20668>, 0x555555810e0c <_PyEval_Fast+17036>, 0x555555810e79 <_PyEval_Fast+17145>, 0x5555558118d7 <_PyEval_Fast+19799>, 0x5555558125a4 <_PyEval_Fast+23076>, 0x555555812537 <_PyEval_Fast+22967>, 0x555555810cc2 <_PyEval_Fast+16706>, 0x555555810c55 <_PyEval_Fast+16597>, 0x555555810be8 <_PyEval_Fast+16488>, 0x555555810b7b <_PyEval_Fast+16379>, 0x555555810b0e <_PyEval_Fast+16270>, 0x555555811dda <_PyEval_Fast+21082>, 0x555555810d2f <_PyEval_Fast+16815>, 0x555555810d9c <_PyEval_Fast+16924>, 0x555555811aec <_PyEval_Fast+20332>, 0x555555812611 <_PyEval_Fast+23185>, 0x555555810ee6 <_PyEval_Fast+17254>, 0x5555558117d0 <_PyEval_Fast+19536>, 0x55555581062f <_PyEval_Fast+15023>, 0x555555811711 <_PyEval_Fast+19345>, 0x555555811982 <_PyEval_Fast+19970>, 0x55555581282e <_PyEval_Fast+23726>, 0x555555811aac <_PyEval_Fast+20268>, 0x5555558116a2 <_PyEval_Fast+19234>, 0x555555811562 <_PyEval_Fast+18914>, 0x55555581163f <_PyEval_Fast+19135>, 0x5555558115ca <_PyEval_Fast+19018>, 0x555555811944 <_PyEval_Fast+19908>, 0x5555558127eb <_PyEval_Fast+23659>, 0x55555581076d <_PyEval_Fast+15341>, 0x555555811d4a <_PyEval_Fast+20938>, 0x555555811da0 <_PyEval_Fast+21024>, 0x555555811cf2 <_PyEval_Fast+20850>, 0x555555811cac <_PyEval_Fast+20780>, 0x555555811481 <_PyEval_Fast+18689>, 0x55555581236c <_PyEval_Fast+22508>, 0x5555558114dc <_PyEval_Fast+18780>, 0x555555812787 <_PyEval_Fast+23559>, 0x555555812743 <_PyEval_Fast+23491>, 0x5555555b5ea2 <_PyEval_Fast.cold>...} pc = 0x5555558d0bcc "\t" opcode = acc = regs = constants = 0x763574ce9f8 tid = #6 0x00005555556a86bf in _PyEval_Eval (pc=, acc=..., tstate=0x5555559f9460) at Python/ceval_meta.c:2788 ret = cargs = {0x1, 0x763584903f0, 0x555555948a80 <_Py_NoneStruct>, 0x555555948a80 <_Py_NoneStruct>, 0x555555948a80 <_Py_NoneStruct>, 0x55555593c080 <_Py_FalseStruct>, 0x76356eb63d0, 0x55555593c040 <_Py_TrueStruct>, 0x1} prevargs = 0x7ffff37a9d08 ret = cargs = prevargs = #7 _PyFunction_Vectorcall (func=0x763575e7130, stack=0x76358420e30, nargsf=, kwnames=) at Python/ceval_meta.c:3185 tstate = 0x5555559f9460 nargs = 1 nkwargs = ret = 0x0 acc = {as_int64 = 1} err = 0 extra = #8 0x00007ffff739ce87 in forward_ndarray_method (self=, args=0x55555594b2d0 <_Py_EmptyTupleStruct+16>, kwds=0x0, forwarding_callable=0x763575e7130) at numpy/core/src/multiarray/methods.c:89 sargs = 0x76358420e10 ret = i = n = #9 0x00005555557bdbfb in method_vectorcall_VARARGS_KEYWORDS (func=0x7635743c6d0, args=0x7ffff37a9d08, nargsf=, kwnames=) at Objects/descrobject.c:349 tstate = 0x5555559f9460 nargs = 1 argstuple = 0x55555594b2d0 <_Py_EmptyTupleStruct+16> result = 0x0 kwdict = 0x0 meth = #10 0x000055555580cdc6 in _PyEval_Fast (ts=0x5555559f9460, initial_acc=..., initial_pc=0x5555558cec6f "\t") at Python/ceval.c:731 --Type for more, q to quit, c to continue without paging-- nargs = 8123251307008 func = nargsf = res = 0x76357823200 frame_delta = 8123251307008 opcode_targets_base = {0x0, 0x55555580e54c <_PyEval_Fast+6604>, 0x55555580e4f0 <_PyEval_Fast+6512>, 0x55555580e4cd <_PyEval_Fast+6477>, 0x55555580e494 <_PyEval_Fast+6420>, 0x55555580e42d <_PyEval_Fast+6317>, 0x55555580f525 <_PyEval_Fast+10661>, 0x55555580ddca <_PyEval_Fast+4682>, 0x5555555b5ea2 <_PyEval_Fast.cold>, 0x55555580cd0f <_PyEval_Fast+399>, 0x55555580cd02 <_PyEval_Fast+386>, 0x55555580dcde <_PyEval_Fast+4446>, 0x55555580fa35 <_PyEval_Fast+11957>, 0x55555580f93d <_PyEval_Fast+11709>, 0x55555580fb3a <_PyEval_Fast+12218>, 0x55555580e1ea <_PyEval_Fast+5738>, 0x55555580e243 <_PyEval_Fast+5827>, 0x55555580e2f5 <_PyEval_Fast+6005>, 0x55555580e350 <_PyEval_Fast+6096>, 0x55555580e29c <_PyEval_Fast+5916>, 0x55555580d9f1 <_PyEval_Fast+3697>, 0x55555580d983 <_PyEval_Fast+3587>, 0x55555580d7d7 <_PyEval_Fast+3159>, 0x55555580d842 <_PyEval_Fast+3266>, 0x55555580e376 <_PyEval_Fast+6134>, 0x55555580d76c <_PyEval_Fast+3052>, 0x55555580d66d <_PyEval_Fast+2797>, 0x55555580d918 <_PyEval_Fast+3480>, 0x55555580d8ad <_PyEval_Fast+3373>, 0x55555580da5c <_PyEval_Fast+3804>, 0x55555580dac7 <_PyEval_Fast+3911>, 0x55555580db32 <_PyEval_Fast+4018>, 0x55555580db9d <_PyEval_Fast+4125>, 0x55555580dc08 <_PyEval_Fast+4232>, 0x55555580e125 <_PyEval_Fast+5541>, 0x55555580e181 <_PyEval_Fast+5633>, 0x55555580e0b6 <_PyEval_Fast+5430>, 0x55555580d4be <_PyEval_Fast+2366>, 0x55555580d453 <_PyEval_Fast+2259>, 0x55555580dc73 <_PyEval_Fast+4339>, 0x55555580d312 <_PyEval_Fast+1938>, 0x55555580d37d <_PyEval_Fast+2045>, 0x55555580d602 <_PyEval_Fast+2690>, 0x55555580d0b8 <_PyEval_Fast+1336>, 0x55555580d11f <_PyEval_Fast+1439>, 0x55555580d18a <_PyEval_Fast+1546>, 0x55555580e789 <_PyEval_Fast+7177>, 0x55555580d3e8 <_PyEval_Fast+2152>, 0x55555580d597 <_PyEval_Fast+2583>, 0x55555580d529 <_PyEval_Fast+2473>, 0x55555580e751 <_PyEval_Fast+7121>, 0x55555580ce6d <_PyEval_Fast+749>, 0x55555580f3af <_PyEval_Fast+10287>, 0x55555580cfc4 <_PyEval_Fast+1092>, 0x55555580cea9 <_PyEval_Fast+809>, 0x55555580d1f5 <_PyEval_Fast+1653>, 0x55555580e58e <_PyEval_Fast+6670>, 0x55555580ded0 <_PyEval_Fast+4944>, 0x55555580e3e1 <_PyEval_Fast+6241>, 0x55555580cf54 <_PyEval_Fast+980>, 0x55555580e6ea <_PyEval_Fast+7018>, 0x55555580e615 <_PyEval_Fast+6805>, 0x55555580e679 <_PyEval_Fast+6905>, 0x55555580e5d6 <_PyEval_Fast+6742>, 0x55555580df0a <_PyEval_Fast+5002>, 0x55555580df4e <_PyEval_Fast+5070>, 0x55555580dfbf <_PyEval_Fast+5183>, 0x55555580df84 <_PyEval_Fast+5124>, 0x55555580e016 <_PyEval_Fast+5270>, 0x55555580e06f <_PyEval_Fast+5359>, 0x55555580f613 <_PyEval_Fast+10899>, 0x55555580f62e <_PyEval_Fast+10926>, 0x55555580f5ad <_PyEval_Fast+10797>, 0x55555580f2b8 <_PyEval_Fast+10040>, 0x55555580f31d <_PyEval_Fast+10141>, 0x55555580f87c <_PyEval_Fast+11516>, 0x55555580ef19 <_PyEval_Fast+9113>, 0x55555580f6e6 <_PyEval_Fast+11110>, 0x55555580f720 <_PyEval_Fast+11168>, 0x55555580f3cd <_PyEval_Fast+10317>, 0x55555580f493 <_PyEval_Fast+10515>, 0x55555580f4e9 <_PyEval_Fast+10601>, 0x55555580ef4d <_PyEval_Fast+9165>, 0x55555580f3f7 <_PyEval_Fast+10359>, 0x55555580f452 <_PyEval_Fast+10450>, 0x55555580cc3e <_PyEval_Fast+190>, 0x55555580e7f4 <_PyEval_Fast+7284>, 0x55555580e8c3 <_PyEval_Fast+7491>, 0x55555580d6d8 <_PyEval_Fast+2904>, 0x55555580d720 <_PyEval_Fast+2976>, 0x55555580de90 <_PyEval_Fast+4880>, 0x55555580ea87 <_PyEval_Fast+7943>, 0x55555580eb7c <_PyEval_Fast+8188>, 0x55555580eab8 <_PyEval_Fast+7992>, 0x55555580eb46 <_PyEval_Fast+8134>, 0x55555580ec04 <_PyEval_Fast+8324>, 0x55555580efb0 <_PyEval_Fast+9264>, 0x55555580f01c <_PyEval_Fast+9372>, 0x55555580f069 <_PyEval_Fast+9449>, 0x55555580eedf <_PyEval_Fast+9055>, 0x55555580e853 <_PyEval_Fast+7379>, 0x55555580e91b <_PyEval_Fast+7579>, 0x55555580e9b1 <_PyEval_Fast+7729>, 0x55555580f16a <_PyEval_Fast+9706>, 0x55555580ea59 <_PyEval_Fast+7897>, 0x55555580ee4e <_PyEval_Fast+8910>, 0x55555580f577 <_PyEval_Fast+10743>, 0x55555580f0c8 <_PyEval_Fast+9544>, 0x55555580f13c <_PyEval_Fast+9660>, 0x55555580f102 <_PyEval_Fast+9602>, 0x55555580ed42 <_PyEval_Fast+8642>, 0x55555580ece9 <_PyEval_Fast+8553>, 0x55555580ed9c <_PyEval_Fast+8732>, 0x55555580edf5 <_PyEval_Fast+8821>, 0x55555580ec8b <_PyEval_Fast+8459>, 0x55555580ec32 <_PyEval_Fast+8370>, 0x55555580f287 <_PyEval_Fast+9991>, 0x55555580f257 <_PyEval_Fast+9943>, 0x55555580f399 <_PyEval_Fast+10265>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x5555558119c9 <_PyEval_Fast+20041>, 0x555555811a08 <_PyEval_Fast+20104>, 0x555555811a29 <_PyEval_Fast+20137>, 0x555555811a60 <_PyEval_Fast+20192>, 0x55555581080f <_PyEval_Fast+15503>, 0x5555555b5ea2 <_PyEval_Fast.cold> , 0x55555581228f <_PyEval_Fast+22287>, 0x5555558122fc <_PyEval_Fast+22396>, 0x555555811f94 <_PyEval_Fast+21524>, 0x555555811f27 <_PyEval_Fast+21415>, 0x5555558123fd <_PyEval_Fast+22653>, 0x555555810700 <_PyEval_Fast+15232>, 0x555555810aa1 <_PyEval_Fast+16161>, 0x5555558107a2 <_PyEval_Fast+15394>, 0x555555812001 <_PyEval_Fast+21633>, 0x5555558120db <_PyEval_Fast+21851>, 0x55555581206e <_PyEval_Fast+21742>, 0x5555558121b5 <_PyEval_Fast+22069>, 0x555555812148 <_PyEval_Fast+21960>, 0x555555812222 <_PyEval_Fast+22178>, 0x5555558124d5 <_PyEval_Fast+22869>, 0x55555581246a <_PyEval_Fast+22762>, 0x555555811c3c <_PyEval_Fast+20668>, 0x555555810e0c <_PyEval_Fast+17036>, 0x555555810e79 <_PyEval_Fast+17145>, 0x5555558118d7 <_PyEval_Fast+19799>, 0x5555558125a4 <_PyEval_Fast+23076>, 0x555555812537 <_PyEval_Fast+22967>, 0x555555810cc2 <_PyEval_Fast+16706>, 0x555555810c55 <_PyEval_Fast+16597>, 0x555555810be8 <_PyEval_Fast+16488>, 0x555555810b7b <_PyEval_Fast+16379>, 0x555555810b0e <_PyEval_Fast+16270>, 0x555555811dda <_PyEval_Fast+21082>, 0x555555810d2f <_PyEval_Fast+16815>, 0x555555810d9c <_PyEval_Fast+16924>, 0x555555811aec <_PyEval_Fast+20332>, 0x555555812611 <_PyEval_Fast+23185>, 0x555555810ee6 <_PyEval_Fast+17254>, 0x5555558117d0 <_PyEval_Fast+19536>, 0x55555581062f <_PyEval_Fast+15023>, 0x555555811711 <_PyEval_Fast+19345>, 0x555555811982 <_PyEval_Fast+19970>, 0x55555581282e <_PyEval_Fast+23726>, 0x555555811aac <_PyEval_Fast+20268>, 0x5555558116a2 <_PyEval_Fast+19234>, 0x555555811562 <_PyEval_Fast+18914>, 0x55555581163f <_PyEval_Fast+19135>, 0x5555558115ca <_PyEval_Fast+19018>, 0x555555811944 <_PyEval_Fast+19908>, 0x5555558127eb <_PyEval_Fast+23659>, 0x55555581076d <_PyEval_Fast+15341>, 0x555555811d4a <_PyEval_Fast+20938>, 0x555555811da0 <_PyEval_Fast+21024>, 0x555555811cf2 <_PyEval_Fast+20850>, 0x555555811cac <_PyEval_Fast+20780>, 0x555555811481 <_PyEval_Fast+18689>, 0x55555581236c <_PyEval_Fast+22508>, 0x5555558114dc <_PyEval_Fast+18780>, 0x555555812787 <_PyEval_Fast+23559>, 0x555555812743 <_PyEval_Fast+23491>, 0x5555555b5ea2 <_PyEval_Fast.cold>...} pc = 0x5555558cec6f "\t" opcode = acc = regs = constants = 0x76356ea2b40 tid = #11 0x00005555556a86bf in _PyEval_Eval (pc=, acc=..., tstate=0x5555559f9460) at Python/ceval_meta.c:2788 ret = cargs = {0x0, 0x763584903f0, 0x55555593c080 <_Py_FalseStruct>, 0x7ffff2fa9ac0, 0x7ffff2fa9b60, 0x4, 0x2, 0x3200000008, 0x0} prevargs = 0x0 ret = cargs = prevargs = #12 _PyFunction_Vectorcall (func=0x76356df0470, stack=0x7ffff37a9da8, nargsf=, kwnames=) at Python/ceval_meta.c:3185 tstate = 0x5555559f9460 nargs = 1 nkwargs = ret = 0x0 acc = {as_int64 = 1} err = 0 extra = #13 0x00005555557b3ab4 in _PyObject_VectorcallTstate (kwnames=0x0, nargsf=1, args=0x7ffff37a9da8, callable=0x76356df0470, tstate=0x5555559f9460) at ./Include/cpython/abstract.h:118 func = res = nargs = #14 method_vectorcall (method=, args=0x55555594b2f0 <_Py_EmptyTupleStruct+48>, nargsf=, kwnames=0x0) at Objects/classobject.c:62 nkwargs = totalargs = newargs_stack = {0x7fffffffd35f, 0x5555559f9460, 0x5555559f9460, 0xc93ca52021d5b00, 0x55555599bc60 <_PyRuntime+640>} newargs = tstate = 0x5555559f9460 self = 0x763568f8e90 func = 0x76356df0470 nargs = 0 result = #15 0x0000555555771a41 in t_bootstrap (boot_raw=0x7635795ba30) at ./Modules/_threadmodule.c:1289 boot = 0x7635795ba30 tstate = 0x5555559f9460 res = --Type for more, q to quit, c to continue without paging-- #16 0x00005555557064bb in pythread_wrapper (arg=) at Python/thread_pthread.h:245 callback = func = 0x5555557719f0 func_arg = 0x7635795ba30 #17 0x00007ffff7d38947 in start_thread (arg=) at pthread_create.c:435 ret = pd = out = unwind_buf = {cancel_jmp_buf = {{jmp_buf = {140737278289472, -7107990009691300924, 140737488343902, 140737488343903, 0, 140737269899264, 7108015344611171268, 7108007434102184900}, mask_was_saved = 0}}, priv = {pad = {0x0, 0x0, 0x0, 0x0}, data = {prev = 0x0, cleanup = 0x0, canceltype = 0}}} not_first_call = #18 0x00007ffff7dc8a44 in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:100 ```

which is a call to NPY_AUXDATA_FREE(auxdata);

Note that, from time to time, I also observe the following exception before a deadlock:

Traceback (most recent call last):
  File "/home/ogrisel/code/nogil/Lib/threading.py", line 935, in _bootstrap_inner
    self.run()
  File "/home/ogrisel/code/nogil/Lib/threading.py", line 886, in run
    self._target(*self._args, **self._kwargs)
  File "/home/ogrisel/threaded_numpy.py", line 15, in worker
    f(item)
  File "/home/ogrisel/threaded_numpy.py", line 7, in f
    return a.max()
  File "/home/ogrisel/nogil-venv/lib/python3.9/site-packages/numpy/core/_methods.py", line 40, in _amax
    return umr_maximum(a, axis, None, out, keepdims, initial, where)
RuntimeError: Identity cache already includes the item.

[colesbury]

@ogrisel what version of NumPy are you using and how did you install it?

There are some parts of NumPy that are not thread-safe without the GIL. I've patched these in my fork of NumPy and the prebuilt binary wheels include the patches, so if you run pip install numpy you should get a version that works without the GIL. However, if you're installing from source you should use the version from my fork, such as https://github.com/colesbury/numpy/tree/v1.19.4-nogil or https://github.com/colesbury/numpy/tree/v1.22.3-nogil.

I'm tracking the packages that require changes in the wiki: https://github.com/colesbury/nogil/wiki/Supported-C-API-Extensions

[colesbury]

@ogrisel never mind - I'm now able to reproduce the issue with the patched NumPy 1.22.3 (I was previously testing with the patched 1.19.4).

Thanks for the bug report and reproducer.

[colesbury]

I've pushed a fix to the nogil NumPy fork and re-built and uploaded the wheels.

You'll need to clear the pip cache because the version number has not changed:

pip uninstall numpy
pip cache purge
pip install numpy

Please let me know if you run into any other multithreading issues with NumPy.

[ogrisel]

I confirm this fixed the reported bug! Thank you very much for the quick fix.