numpy.linalg and multiprocessing crash (Trac #2201)

[numpy-gitbot]

Original ticket http://projects.scipy.org/numpy/ticket/2201 on 2012-08-14 by trac user agchang, assigned to atmention:pv.

There seems to be an issue with numpy.linalg being called in unison with python's multiprocessing module. When invoking a numpy.linalg method from within a subprocess, e.g., through a worker pool and map() function, the program hangs and ignores all interrupt signals.

The code provided below will demonstrate the case:

import numpy as np
import multiprocessing as mp

def foo(x):
    print np.linalg.inv([[2,3],[2,2]]) #this causes the crash
    #print np.dot([[1,2],[3,4]],[[1,2],[3,4]]) # this works fine

def test():
    print "running..."
    print np.__version__
    print mp.__version__
    vals = [1,2,3,4]
    #pool = mp.Pool(1) #This has an issue
    #pool = mp.Pool(mp.cpu_count()) #this has an issue
    pool.map(foo, vals)

if __name__ == "__main__":
    test() 
    #foo(1) #this works fine

By hang I mean the program becomes unresponsive and does not respond to interrupts(Ctrl-C).

I took a look using pdb and it seems that it hangs after the call waiter.acquire() in threading.py(the python system module) so I suspect some sort of deadlock?

multiprocessing.version is 0.70a1 numpy.version is 1.6.1

Some potentially related packages?

ii libblas3gf 1.2.20110419-2 Basic Linear Algebra Reference implementatio ii liblapack3gf 3.3.1-1 library of linear algebra routines 3 - share ii python 2.7.3-0ubuntu2 interactive high-level object-oriented langu ii python-numpy 1:1.6.1-6ubunt Numerical Python adds a fast array facility

My kernel version is(uname -a):

Linux agc 3.2.0-29-generic #46-Ubuntu SMP Fri Jul 27 17:03:23 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

Let me know for any additional information.

Thanks.

[numpy-gitbot]

atmention:pv wrote on 2012-08-14

I cannot reproduce this (Ubuntu x86, OpenSUSE x86_64, both with Numpy 1.6.1).

[numpy-gitbot]

trac user agchang wrote on 2012-08-14

Sorry, I forgot to mention something very relevant. This bug seems to occur only on specific hardware, i.e, Macbook Pro(Mid-2010). I ran this also on a non Macintosh machine and it was able to work. Also, I'm not sure how relevant this ticket it is, but it seems to have similar problems as well.

http://projects.scipy.org/numpy/ticket/2091

However, this person uses OSX so it is presumably on Macintosh hardware as well.

[numpy-gitbot]

trac user agchang wrote on 2012-08-14

The machine that it works on(the non-Macintosh) has a kernel version of:

Linux byrd 3.2.0-27-generic #43-Ubuntu SMP Fri Jul 6 14:25:57 UTC 2012 x86_64 x86_64 x86_64 GNU/Linux

[numpy-gitbot]

trac user agchang wrote on 2012-08-15

This also seems to run on Mac OSX Snow Leopard my machine as well.

[numpy-gitbot]

atmention:pv wrote on 2012-08-15

This is probably relevant: http://mail.scipy.org/pipermail/numpy-discussion/2012-August/063589.html

If this is indeed the origin of this crash, then it's an Apple issue with multiprocessing, and not fixable by us (except by shipping binaries with Accelerate disabled, but that's a workaround).

[numpy-gitbot]

trac user agchang wrote on 2012-08-22

Hm. Sounds like it. Would it still make sense that it works on Mac OSX Snow Leopard on my machine, but not Ubuntu? Maybe Ubuntu does utilize the Accelerate framework while Mac OSX does not, by default?

[numpy-gitbot]

atmention:pv wrote on 2012-08-22

I must have misread this bug report originally. Accelerate is Apple software that comes with OSX, so I don't think running Ubuntu on the hardware uses it. So this seems to be a different issue than #2091