shunwang / numexpr

Automatically exported from code.google.com/p/numexpr
MIT License
0 stars 0 forks source link

numexpr unstable due to use of platform.machine() #123

Open GoogleCodeExporter opened 9 years ago

GoogleCodeExporter commented 9 years ago
Upon import of numexpr on certain systems, the following exception is thrown: 
IOError: [Errno 10] No child processes

For my application, this is occurring when numexpr is used with PySpark(which 
in turn uses Py4J), however the use of platform.machine() may cause trouble due 
to its use of os.popen

What steps will reproduce the problem?
1. Install PySpark
2. Run the attached script via "pyspark Numexpr_test.py" 
3.

What is the expected output? What do you see instead?
The script should print two lists of ndarrays, however only 'arrayRDD' is 
printed, and then the script dies. Much of the error log is from PySpark,
but the core Python stack dump is here:  
File 
"/opt/devel/moretools/anaconda/lib/python2.7/site-packages/numexpr/__init__.py",
 line 63, in <module>
    if 'sparc' in platform.machine():
  File "/opt/devel/moretools/anaconda/lib/python2.7/platform.py", line 1347, in machine
    return uname()[4]
  File "/opt/devel/moretools/anaconda/lib/python2.7/platform.py", line 1277, in uname
    processor = _syscmd_uname('-p','')
  File "/opt/devel/moretools/anaconda/lib/python2.7/platform.py", line 1012, in _syscmd_uname
    rc = f.close()
IOError: [Errno 10] No child processes

Indicating that the function 'returnScaledNdarray' threw an exception upon 
import of numexpr.  

What version of the product are you using? On what operating system?
numexpr 2.2.2, provided through Anaconda, running on Ubuntu 12.04 x86_64. 
However note the offending code is being run through Py4J. 

Please provide any additional information below.
The offending code in present in __init__.py, beginning with line 63 (in 
2.2.2): 
       if 'sparc' in platform.machine():

If I remove the if statement and simply set:
       set_num_threads(nthreads)
and thus never reference platform.machine() the problem does not occur. 

Original issue reported on code.google.com by michael....@gmail.com on 10 Dec 2013 at 3:18

Attachments: