search

radical-cybertools / MTMS

0 stars 1 forks source link

Error while installing via setup.py #1

Open vivek-bala opened 10 years ago

vivek-bala commented 10 years ago

Error in atexit._run_exitfuncs: Traceback (most recent call last): File "/opt/apps/python/epd/7.3.2/lib/python2.7/atexit.py", line 24, in _run_exitfuncs func(_targs, _kargs) File "/opt/apps/python/epd/7.3.2/lib/python2.7/multiprocessing/util.py", line 284, in _exit_function info('process shutting down') TypeError: 'NoneType' object is not callable Error in sys.exitfunc: Traceback (most recent call last): File "/opt/apps/python/epd/7.3.2/lib/python2.7/atexit.py", line 24, in _run_exitfuncs func(_targs, _kargs) File "/opt/apps/python/epd/7.3.2/lib/python2.7/multiprocessing/util.py", line 284, in _exit_function info('process shutting down') TypeError: 'NoneType' object is not callable

andre-merzky commented 10 years ago

Hi Vivek,

please make sure that you use a Python version newer than 2.7 -- one source of this error to appear is an older python version, such as the ones pre-installed on MacOS... Please let us know though if the python version is in fact newer, then we'll check this out in more detail!

Thanks! :)

oleweidner commented 10 years ago

I believe the "epd/7.3.2” is the only usable Python version on stampede. I’m a little surprised that we all the sudden have version requirements for Python > 2.7? As of now I was under the impression that we’re said >= 2.6 — I think that’s what most of the documentation(s) say as well.

The version requirements imposed by you doesn't seem realistic to me and I can already tell you that it will create a great deal of grief / problems with our users and is hence IMHO not acceptable. Can you please elaborate on the problem? Can it be alleviated by simplifying installer scripts? I have been installing RP and SP and HT-BAC repeatedly on Python 2.6 as well as the specific 2.7 version on stampede Vivek seems to be using.

Vivek: can you please give use more informations about your installation procedure (i.e., full command lines) as well as the full output?

Thanks. Ole

On May 16, 2014, at 7:21 PM, Andre Merzky notifications@github.com wrote:

Hi Vivek,

please make sure that you use a Python version newer than 2.7 -- one source of this error to appear is an older python version, such as the ones pre-installed on MacOS... Please let us know though if the python version is in fact newer, then we'll check this out in more detail!

Thanks! :)

— Reply to this email directly or view it on GitHub.

vivek-bala commented 10 years ago

Like Ole mentioned, this is python 2.7.3. (EPD/7.3.2) and there is only one version available on Stampede. The RP documentation says python >= 2.5, so I guess we might need to make some features back compatible in a sense.

Full CL and output :-

login4$ python virtualenv-1.11/virtualenv.py $HOME/mtms
PYTHONHOME is set.  You *must* activate the virtualenv before using it
New python executable in /home1/02734/vivek91/mtms/bin/python
Installing setuptools, pip...done.
login4$ source $HOME/mtms/bin/activate
(mtms)login4$ git clone https://github.com/radical-cybertools/MTMS.git
Initialized empty Git repository in /home1/02734/vivek91/MTMS/.git/
remote: Counting objects: 357, done.
remote: Compressing objects: 100% (170/170), done.
remote: Total 357 (delta 163), reused 353 (delta 161)
Receiving objects: 100% (357/357), 13.21 MiB | 717 KiB/s, done.
Resolving deltas: 100% (163/163), done.
(mtms)login4$ cd MTMS
(mtms)login4$ python setup.py install

running install
running bdist_egg
running egg_info
creating src/radical.ensemblemd.mtms.egg-info
writing requirements to src/radical.ensemblemd.mtms.egg-info/requires.txt
writing src/radical.ensemblemd.mtms.egg-info/PKG-INFO
writing namespace_packages to src/radical.ensemblemd.mtms.egg-info/namespace_packages.txt
writing top-level names to src/radical.ensemblemd.mtms.egg-info/top_level.txt
writing dependency_links to src/radical.ensemblemd.mtms.egg-info/dependency_links.txt
writing manifest file 'src/radical.ensemblemd.mtms.egg-info/SOURCES.txt'
reading manifest file 'src/radical.ensemblemd.mtms.egg-info/SOURCES.txt'
writing manifest file 'src/radical.ensemblemd.mtms.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-x86_64/egg
running install_lib
running build_py
creating build
creating build/lib
creating build/lib/radical
copying src/radical/__init__.py -> build/lib/radical
creating build/lib/radical/ensemblemd
copying src/radical/ensemblemd/__init__.py -> build/lib/radical/ensemblemd
creating build/lib/radical/ensemblemd/mtms
copying src/radical/ensemblemd/mtms/mtms.py -> build/lib/radical/ensemblemd/mtms
copying src/radical/ensemblemd/mtms/version.py -> build/lib/radical/ensemblemd/mtms
copying src/radical/ensemblemd/mtms/__init__.py -> build/lib/radical/ensemblemd/mtms
copying src/radical/ensemblemd/mtms/VERSION -> build/lib/radical/ensemblemd/mtms
creating build/bdist.linux-x86_64
creating build/bdist.linux-x86_64/egg
creating build/bdist.linux-x86_64/egg/radical
creating build/bdist.linux-x86_64/egg/radical/ensemblemd
creating build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms
copying build/lib/radical/ensemblemd/mtms/VERSION -> build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms
copying build/lib/radical/ensemblemd/mtms/mtms.py -> build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms
copying build/lib/radical/ensemblemd/mtms/version.py -> build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms
copying build/lib/radical/ensemblemd/mtms/__init__.py -> build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms
copying build/lib/radical/ensemblemd/__init__.py -> build/bdist.linux-x86_64/egg/radical/ensemblemd
copying build/lib/radical/__init__.py -> build/bdist.linux-x86_64/egg/radical
byte-compiling build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms/mtms.py to mtms.pyc
byte-compiling build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms/version.py to version.pyc
byte-compiling build/bdist.linux-x86_64/egg/radical/ensemblemd/mtms/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/radical/ensemblemd/__init__.py to __init__.pyc
byte-compiling build/bdist.linux-x86_64/egg/radical/__init__.py to __init__.pyc
creating build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/PKG-INFO -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/SOURCES.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/dependency_links.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/namespace_packages.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/not-zip-safe -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/requires.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
copying src/radical.ensemblemd.mtms.egg-info/top_level.txt -> build/bdist.linux-x86_64/egg/EGG-INFO
creating dist
creating 'dist/radical.ensemblemd.mtms-0.3.0-py2.7.egg' and adding 'build/bdist.linux-x86_64/egg' to it
removing 'build/bdist.linux-x86_64/egg' (and everything under it)
Processing radical.ensemblemd.mtms-0.3.0-py2.7.egg
creating /home1/02734/vivek91/mtms/lib/python2.7/site-packages/radical.ensemblemd.mtms-0.3.0-py2.7.egg
Extracting radical.ensemblemd.mtms-0.3.0-py2.7.egg to /home1/02734/vivek91/mtms/lib/python2.7/site-packages
Adding radical.ensemblemd.mtms 0.3.0 to easy-install.pth file

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/radical.ensemblemd.mtms-0.3.0-py2.7.egg
Processing dependencies for radical.ensemblemd.mtms==0.3.0
Searching for radical.pilot
Reading https://pypi.python.org/simple/radical.pilot/
Best match: radical.pilot 0.12
Downloading https://pypi.python.org/packages/source/r/radical.pilot/radical.pilot-0.12.tar.gz#md5=afd930abd5111e8776e73acdbadf1d52
Processing radical.pilot-0.12.tar.gz
Writing /tmp/easy_install-fHghPh/radical.pilot-0.12/setup.cfg
Running radical.pilot-0.12/setup.py -q bdist_egg --dist-dir /tmp/easy_install-fHghPh/radical.pilot-0.12/egg-dist-tmp-b8ByCS
version: 0.12 (fatal: Not a git repository (or any of the parent directories): .git
fatal: Not a git repository (or any of the parent directories): .git
@)
creating /tmp/easy_install-fHghPh/radical.pilot-0.12/src/radical/pilot//VERSION
Adding radical.pilot 0.12 to easy-install.pth file
Installing radicalpilot-version script to /home1/02734/vivek91/mtms/bin
Installing bootstrap-and-run-agent script to /home1/02734/vivek91/mtms/bin
Installing radicalpilot-profiler script to /home1/02734/vivek91/mtms/bin

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/radical.pilot-0.12-py2.7.egg
Searching for python-hostlist
Reading https://pypi.python.org/simple/python-hostlist/
Reading http://www.nsc.liu.se/~kent/python-hostlist/
Best match: python-hostlist 1.14
Downloading https://www.nsc.liu.se/~kent/python-hostlist/python-hostlist-1.14.tar.gz
Processing python-hostlist-1.14.tar.gz
Writing /tmp/easy_install-oCkP03/python-hostlist-1.14/setup.cfg
Running python-hostlist-1.14/setup.py -q bdist_egg --dist-dir /tmp/easy_install-oCkP03/python-hostlist-1.14/egg-dist-tmp-eaogyx
zip_safe flag not set; analyzing archive contents...
Adding python-hostlist 1.14 to easy-install.pth file
Installing dbuck script to /home1/02734/vivek91/mtms/bin
Installing hostlist script to /home1/02734/vivek91/mtms/bin
Installing pshbak script to /home1/02734/vivek91/mtms/bin
Installing hostgrep script to /home1/02734/vivek91/mtms/bin

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/python_hostlist-1.14-py2.7.egg
Searching for pymongo>=2.5
Reading https://pypi.python.org/simple/pymongo/
Best match: pymongo 2.7
Downloading https://pypi.python.org/packages/source/p/pymongo/pymongo-2.7.tar.gz#md5=8d7f8517b4f979bae531fa81dae6215e
Processing pymongo-2.7.tar.gz
Writing /tmp/easy_install-FpGk56/pymongo-2.7/setup.cfg
Running pymongo-2.7/setup.py -q bdist_egg --dist-dir /tmp/easy_install-FpGk56/pymongo-2.7/egg-dist-tmp-iRvmTG
zip_safe flag not set; analyzing archive contents...
Adding pymongo 2.7 to easy-install.pth file

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/pymongo-2.7-py2.7-linux-x86_64.egg
Searching for radical.utils
Reading https://pypi.python.org/simple/radical.utils/
Best match: radical.utils 0.7.3
Downloading https://pypi.python.org/packages/source/r/radical.utils/radical.utils-0.7.3.tar.gz#md5=263e64d1e003de47476bc3b27efa3f06
Processing radical.utils-0.7.3.tar.gz
Writing /tmp/easy_install-a84MMk/radical.utils-0.7.3/setup.cfg
Running radical.utils-0.7.3/setup.py -q bdist_egg --dist-dir /tmp/easy_install-a84MMk/radical.utils-0.7.3/egg-dist-tmp-WzvtRe
version: 0.7.3 (v0.7.3)
creating /tmp/easy_install-a84MMk/radical.utils-0.7.3/radical/utils/VERSION
Adding radical.utils 0.7.3 to easy-install.pth file
Installing dump_mongodb.py script to /home1/02734/vivek91/mtms/bin

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/radical.utils-0.7.3-py2.7.egg
Searching for saga-python
Reading https://pypi.python.org/simple/saga-python/
Best match: saga-python 0.14
Downloading https://pypi.python.org/packages/source/s/saga-python/saga-python-0.14.tar.gz#md5=02f93712296bc8ed569d3fc14c971ba0
Processing saga-python-0.14.tar.gz
Writing /tmp/easy_install-k2erpj/saga-python-0.14/setup.cfg
Running saga-python-0.14/setup.py -q bdist_egg --dist-dir /tmp/easy_install-k2erpj/saga-python-0.14/egg-dist-tmp-UbpfYF
reading  /tmp/easy_install-k2erpj/saga-python-0.14/VERSION
creating /tmp/easy_install-k2erpj/saga-python-0.14/VERSION
creating /tmp/easy_install-k2erpj/saga-python-0.14/saga/VERSION
Adding saga-python 0.14 to easy-install.pth file

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/saga_python-0.14-py2.7.egg
Searching for colorama
Reading https://pypi.python.org/simple/colorama/
Best match: colorama 0.3.1
Downloading https://pypi.python.org/packages/source/c/colorama/colorama-0.3.1.zip#md5=54038459dba206a8af3db085be907d08
Processing colorama-0.3.1.zip
Writing /tmp/easy_install-JWSN11/colorama-0.3.1/setup.cfg
Running colorama-0.3.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-JWSN11/colorama-0.3.1/egg-dist-tmp-I5i7K2
zip_safe flag not set; analyzing archive contents...
Adding colorama 0.3.1 to easy-install.pth file

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/colorama-0.3.1-py2.7.egg
Searching for apache-libcloud
Reading https://pypi.python.org/simple/apache-libcloud/
Best match: apache-libcloud 0.14.1
Downloading https://pypi.python.org/packages/source/a/apache-libcloud/apache-libcloud-0.14.1.zip#md5=083c93523b1c7b29c2ae6b96864f6a5a
Processing apache-libcloud-0.14.1.zip
Writing /tmp/easy_install-v1HaRB/apache-libcloud-0.14.1/setup.cfg
Running apache-libcloud-0.14.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-v1HaRB/apache-libcloud-0.14.1/egg-dist-tmp-VrJ7m_
no previously-included directories found matching 'libcloud/test/secrets.py'
Adding apache-libcloud 0.14.1 to easy-install.pth file

Installed /home1/02734/vivek91/mtms/lib/python2.7/site-packages/apache_libcloud-0.14.1-py2.7.egg
Searching for setuptools==2.0.2
Best match: setuptools 2.0.2
Adding setuptools 2.0.2 to easy-install.pth file
Installing easy_install-3.4 script to /home1/02734/vivek91/mtms/bin
Installing easy_install script to /home1/02734/vivek91/mtms/bin

Using /home1/02734/vivek91/mtms/lib/python2.7/site-packages
Finished processing dependencies for radical.ensemblemd.mtms==0.3.0
Error in atexit._run_exitfuncs:
Traceback (most recent call last):
  File "/opt/apps/python/epd/7.3.2/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/opt/apps/python/epd/7.3.2/lib/python2.7/multiprocessing/util.py", line 284, in _exit_function
    info('process shutting down')
TypeError: 'NoneType' object is not callable
Error in sys.exitfunc:
Traceback (most recent call last):
  File "/opt/apps/python/epd/7.3.2/lib/python2.7/atexit.py", line 24, in _run_exitfuncs
    func(*targs, **kargs)
  File "/opt/apps/python/epd/7.3.2/lib/python2.7/multiprocessing/util.py", line 284, in _exit_function
    info('process shutting down')
TypeError: 'NoneType' object is not callable
(mtms)login4$
andre-merzky commented 10 years ago

This has nothing to do with how I changes setup.py. GOogle for the error and 'multiprocessing', and the first hit will point to http://bugs.python.org/issue15881 Its a MP bug which is only fixed in 7.2 ...

I am not sure what 'epd/7.3.2' means -- what python version is that?

Cheers, Andre.

On Sat, May 17, 2014 at 4:17 AM, Ole Weidner notifications@github.com wrote:

I believe the "epd/7.3.2” is the only usable Python version on stampede. I’m a little surprised that we all the sudden have version requirements for Python > 2.7? As of now I was under the impression that we’re said >= 2.6 — I think that’s what most of the documentation(s) say as well.

The version requirements imposed by you doesn't seem realistic to me and I can already tell you that it will create a great deal of grief / problems with our users and is hence IMHO not acceptable. Can you please elaborate on the problem? Can it be alleviated by simplifying installer scripts? I have been installing RP and SP and HT-BAC repeatedly on Python 2.6 as well as the specific 2.7 version on stampede Vivek seems to be using.

Vivek: can you please give use more informations about your installation procedure (i.e., full command lines) as well as the full output?

Thanks. Ole

On May 16, 2014, at 7:21 PM, Andre Merzky notifications@github.com wrote:

Hi Vivek,

please make sure that you use a Python version newer than 2.7 -- one source of this error to appear is an older python version, such as the ones pre-installed on MacOS... Please let us know though if the python version is in fact newer, then we'll check this out in more detail!

Thanks! :)

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub.

It was a sad and disappointing day when I discovered that my Universal Remote Control did not, in fact, control the Universe. (Not even remotely.)

oweidner commented 10 years ago 
login2$ module load python
login2$ which python
/opt/apps/python/epd/7.3.2/bin/python
login2$ python -V
Python 2.7.3 -- EPD 7.3-2 (64-bit)

BTW: This version of Python on stampede is the only one that has support for numpy and scipy which would be otherwise impossible to install.

Vivek: it seems that the installer fails at the very end. Despite the error message, did you check if the installation was successful? Also, could you try with a different virtualenv or a different installation method e.g.,

easy_install.

or 

pip install .

If that succeeds, we can just document this in MTMS.

Mark: did you encounter a similar error on stampede?

marksantcroos commented 10 years ago

On 17 May 2014, at 16:09 , Ole Weidner notifications@github.com wrote:

If that succeeds, we can just document this in MTMS.

Mark: did you encounter a similar error on stampede?

Nope, but didn’t try either.

Vivek: Why are you trying to run this on the stampede head node? In general I suggest not to encourage people to run these things on the head node.