shunwang / numexpr

Automatically exported from code.google.com/p/numexpr
MIT License
0 stars 0 forks source link

Numexpr does not work correctly when run in sub-processes #33

Closed GoogleCodeExporter closed 9 years ago

GoogleCodeExporter commented 9 years ago
The recent support for threads does not behave well when running numexpr in 
sub-processes.

The next code snippet reproduces the problem:

{{{
import numexpr as ne
import numpy as np
import multiprocessing as mp

def _worker(qout = None):
    ra = np.arange(1e3)
    rows = ne.evaluate('ra > 0')
    print "Succeeded in evaluation!\n"

    if qout is not None:
            qout.put("Done")

if __name__ == "__main__":
    #print ne.set_num_threads(1)
    print "**** Running from main process:"
    _worker()
    print "**** Running from subprocess:"
    qout = mp.Queue()
    ps = mp.Process(target=_worker, args=(qout,))
    ps.daemon = True
    ps.start()

    print qout.get()
}}}

which has the next output

{{{
**** Running from main process:
Succeeded in evaluation!

**** Running from subprocess:
[waits forever!]
}}}

Original issue reported on code.google.com by fal...@gmail.com on 27 Sep 2010 at 9:33

GoogleCodeExporter commented 9 years ago
Workaround added in r239.  Initializing the threads whenever Numexpr 
interpreter is run.  I'm leaving this ticket open until I find a way to 
determine when Numexpr really needs to initialize the threads (i.e. avoid to 
always initialize threads).

Original comment by fal...@gmail.com on 27 Sep 2010 at 9:44

GoogleCodeExporter commented 9 years ago
Finally find an elegant fix for this in r240.  Closing ticket.

Original comment by fal...@gmail.com on 30 Sep 2010 at 9:58

GoogleCodeExporter commented 9 years ago
And r241 fixes a couple of typos in r240 :-/

Original comment by fal...@gmail.com on 30 Sep 2010 at 10:00