Still unable to install cactus

[rsharris]

(previously posted this as an add on to issue #41, Oct/6, but perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules' rm -f /afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment python /afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py /afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python New python executable in /afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python Could not find platform dependent libraries <exec_prefix> Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in <module> __boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 548, in <module> main() File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 520, in get_config_var return get_config_vars().get(name) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python is not functioning ERROR: It thinks sys.prefix is u'/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules' (should be u'/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python') ERROR: virtualenv is not compatible with this system or executable make[1]: *** [virtPyRule] Error 100 make[1]: Leaving directory/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules' make: *\ [all] Error 2

[glennhickey]

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6, but perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' rm -f /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment python /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python New python executable in /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 548, in main() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 520, in get_config_var return get_config_vars().get(name) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python is not functioning ERROR: It thinks sys.prefix is u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules' (should be u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python ') ERROR: virtualenv is not compatible with this system or executable make[1]: **\* [virtPyRule] Error 100 make[1]: Leaving directory/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' make: **\* [all] Error 2 — Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46.

[rsharris]

Turns out we aren't using Ubuntu. But looking at the first link, second answer, I'm guessing my virtualenv is too old. It is version 1.5.something, and this site advises not using anything before 1.9: https://virtualenv.pypa.io/en/latest/installation.html

I think I'll have to get an admin to do the virtualenv install. Before I go down that path, do you think that's a good path to head down? Is there a (known) expectation by the cactus package of virtualenv version (I realize no one else may have run into such a problem).

Thanks, Bob H

On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote:

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6, but perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' rm -f /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment python /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python New python executable in /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 548, in main() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 520, in get_config_var return get_config_vars().get(name) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python is not functioning ERROR: It thinks sys.prefix is u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules' (should be u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python ') ERROR: virtualenv is not compatible with this system or executable make[1]: **\* [virtPyRule] Error 100 make[1]: Leaving directory/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' make: **\* [all] Error 2 — Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46. — Reply to this email directly or view it on GitHub.

[glennhickey]

I see no reason not to update the virtualenv submodule in progressive cactus.

Can you please verify that doing so would fix your issue though?

should be as simple as

cd submodules/virtualenv git checkout master git pull

thanks! -Glenn

On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com wrote:

Turns out we aren't using Ubuntu. But looking at the first link, second answer, I'm guessing my virtualenv is too old. It is version 1.5.something, and this site advises not using anything before 1.9: https://virtualenv.pypa.io/en/latest/installation.html

I think I'll have to get an admin to do the virtualenv install. Before I go down that path, do you think that's a good path to head down? Is there a (known) expectation by the cactus package of virtualenv version (I realize no one else may have run into such a problem).

Thanks, Bob H

On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote:

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6, but

perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' rm -f /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment python /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python New python executable in /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 548, in main() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ", line 520, in get_config_var return get_config_vars().get(name) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python is not functioning ERROR: It thinks sys.prefix is u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' (should be u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python ') ERROR: virtualenv is not compatible with this system or executable make[1]: **\* [virtPyRule] Error 100 make[1]: Leaving directory/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' make: **\* [all] Error 2 — Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627 .

[rsharris]

No luck.

Reponse from those commands: Previous HEAD position was 239d5a9... Merge branch 'release/1.8.2' Branch master set up to track remote branch master from origin. Switched to a new branch 'master' baker:virtualenv$ git pull remote: Counting objects: 8, done. remote: Total 8 (delta 3), reused 3 (delta 3), pack-reused 5 Unpacking objects: 100% (8/8), done. From git://github.com/pypa/virtualenv 1076aff..cba89a1 develop -> origin/develop Already up-to-date.

And then a make (after cd back to the progressiveCactus directory) give the same output as before, except that it now reports overwriting the progressiveCactus site.py: Overwriting ...path.../progressiveCactus/python/Python-2.7.10/Lib/site.py with new content The rest of the output from make is identical to before.

As I noted earlier, firing up python from the shell the _sysconfigdata import is OK. This however is using the python I built and installed, not the one inside progressiveCactus. The error log indicates that's the python the exceptions are occurring in. But then right after that something (I can't figure out what) appears to be reporting a problem with the progressiveCactus python. Maybe I'm misinterpreting that.

Earlier in the log it tells me it has installed "New python executable" and gives me the path to the progressiveCactus python. Is the problem that the subsequent steps (the ones that fail) are supposed to be using the progressiveCactus python but for some reason are using the wrong one? --OR-- is the message about $PYTHONHOME and the prefix related to the progressiveCactus python instead of the python I would normally be running?

Thanks, Bob H

On Oct 12, 2015, at 4:40 PM, Glenn Hickey wrote:

I see no reason not to update the virtualenv submodule in progressive cactus.

Can you please verify that doing so would fix your issue though?

should be as simple as

cd submodules/virtualenv git checkout master git pull

thanks! -Glenn

On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com wrote:

Turns out we aren't using Ubuntu. But looking at the first link, second answer, I'm guessing my virtualenv is too old. It is version 1.5.something, and this site advises not using anything before 1.9: https://virtualenv.pypa.io/en/latest/installation.html

I think I'll have to get an admin to do the virtualenv install. Before I go down that path, do you think that's a good path to head down? Is there a (known) expectation by the cactus package of virtualenv version (I realize no one else may have run into such a problem).

Thanks, Bob H

On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote:

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6, but

perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' rm -f /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment python /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python New python executable in /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 548, in main() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ", line 520, in get_config_var return get_config_vars().get(name) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python is not functioning ERROR: It thinks sys.prefix is u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' (should be u'/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python ') ERROR: virtualenv is not compatible with this system or executable make[1]: **\* [virtPyRule] Error 100 make[1]: Leaving directory/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules ' make: **\* [all] Error 2 — Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46. — Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627 .

— Reply to this email directly or view it on GitHub.

[rsharris]

Clarification. By "The error log indicates that's the python the exceptions are occurring in" I meant the python I installed. But when I try the import from that same python launched from the shell, it is successful.

[glennhickey]

I'm not sure. For what it's worth, these are the commands in the Makefile for this part:

python $(PWD)/virtualenv/virtualenv.py ${virtPyDir} . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install . ${virtPyEnv} && cd $(PWD)/psutil/ && ${virtPy} setup.py install . ${virtPyEnv} && cd $(PWD)/biopython/ && echo y | ${virtPy} setup.py install | cat cp ${virtPyEnv} ${myEnv} echo export PYTHONPATH=$(PWD):\$$PYTHONPATH >> ${myEnv} echo export PATH=${virtPyDir}/bin:\$$PATH >> ${myEnv}

On our server the (relevant) part of the output looks like:

rm -f /cluster/home/hickey/genomes/progressiveCactus/submodules/../environment python /cluster/home/hickey/genomes/progressiveCactus/submodules/virtualenv/virtualenv.py /cluster/home/hickey/genomes/progressiveCactus/python New python executable in /cluster/home/hickey/genomes/progressiveCactus/python/bin/python Installing setuptools............done. Installing pip...............done. . /cluster/home/hickey/genomes/progressiveCactus/python/bin/activate && cd /cluster/home/hickey/genomes/progressiveCactus/submodules/networkx && /cluster/home/hicke\ y/genomes/progressiveCactus/python/bin/python setup.py install running install running build running build_py running install_lib creating /cluster/home/hickey/genomes/progressiveCactus/python/lib/python2.7/site-packages/networkx (etc)

From your log, I think it's failing on the second step, installing networkx in the virtual python:

. ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install

Note that ${virPyEnv} is progressiveCactus/environment, and made by a combination of submodules/Makefile and virtualenv.py.

This is the environment that's loaded to switch to the cactus python, and would be culprit if somehow your pythonhome is getting mixed up. I think maybe trying to reproduce with commands outside the Makefile might be the way to go... Wish I could reproduce here to help more.

On Mon, Oct 12, 2015 at 5:05 PM, rsharris notifications@github.com wrote:

No luck.

Reponse from those commands: Previous HEAD position was 239d5a9... Merge branch 'release/1.8.2' Branch master set up to track remote branch master from origin. Switched to a new branch 'master' baker:virtualenv$ git pull remote: Counting objects: 8, done. remote: Total 8 (delta 3), reused 3 (delta 3), pack-reused 5 Unpacking objects: 100% (8/8), done. From git://github.com/pypa/virtualenv 1076aff..cba89a1 develop -> origin/develop Already up-to-date.

And then a make (after cd back to the progressiveCactus directory) give the same output as before, except that it now reports overwriting the progressiveCactus site.py: Overwriting ...path.../progressiveCactus/python/Python-2.7.10/Lib/site.py with new content The rest of the output from make is identical to before.

As I noted earlier, firing up python from the shell the _sysconfigdata import is OK. This however is using the python I built and installed, not the one inside progressiveCactus. The error log indicates that's the python the exceptions are occurring in. But then right after that something (I can't figure out what) appears to be reporting a problem with the progressiveCactus python. Maybe I'm misinterpreting that.

Earlier in the log it tells me it has installed "New python executable" and gives me the path to the progressiveCactus python. Is the problem that the subsequent steps (the ones that fail) are supposed to be using the progressiveCactus python but for some reason are using the wrong one? --OR-- is the message about $PYTHONHOME and the prefix related to the progressiveCactus python instead of the python I would normally be running?

Thanks, Bob H

On Oct 12, 2015, at 4:40 PM, Glenn Hickey wrote:

I see no reason not to update the virtualenv submodule in progressive cactus.

Can you please verify that doing so would fix your issue though?

should be as simple as

cd submodules/virtualenv git checkout master git pull

thanks! -Glenn

On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com wrote:

Turns out we aren't using Ubuntu. But looking at the first link, second answer, I'm guessing my virtualenv is too old. It is version 1.5.something, and this site advises not using anything before 1.9: https://virtualenv.pypa.io/en/latest/installation.html

I think I'll have to get an admin to do the virtualenv install. Before I go down that path, do you think that's a good path to head down? Is there a (known) expectation by the cactus package of virtualenv version (I realize no one else may have run into such a problem).

Thanks, Bob H

On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote:

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6,

but

perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules

' rm -f /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment

python /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py

/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python

New python executable in /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 548, in main() File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 520, in get_config_var return get_config_vars().get(name) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python

is not functioning ERROR: It thinks sys.prefix is u'/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules '

(should be u'/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python

') ERROR: virtualenv is not compatible with this system or executable make[1]: *\ [virtPyRule] Error 100 make[1]: Leaving directory/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules

' make: *\ [all] Error 2

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub < https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147522813 .

[rsharris]

Cool, thanks. I will try more in the morning.

Bob H

On Oct 12, 2015, at 5:28 PM, Glenn Hickey wrote:

I'm not sure. For what it's worth, these are the commands in the Makefile for this part:

python $(PWD)/virtualenv/virtualenv.py ${virtPyDir} . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install . ${virtPyEnv} && cd $(PWD)/psutil/ && ${virtPy} setup.py install . ${virtPyEnv} && cd $(PWD)/biopython/ && echo y | ${virtPy} setup.py install | cat cp ${virtPyEnv} ${myEnv} echo export PYTHONPATH=$(PWD):\$$PYTHONPATH >> ${myEnv} echo export PATH=${virtPyDir}/bin:\$$PATH >> ${myEnv}

On our server the (relevant) part of the output looks like:

rm -f /cluster/home/hickey/genomes/progressiveCactus/submodules/../environment python /cluster/home/hickey/genomes/progressiveCactus/submodules/virtualenv/virtualenv.py /cluster/home/hickey/genomes/progressiveCactus/python New python executable in /cluster/home/hickey/genomes/progressiveCactus/python/bin/python Installing setuptools............done. Installing pip...............done. . /cluster/home/hickey/genomes/progressiveCactus/python/bin/activate && cd /cluster/home/hickey/genomes/progressiveCactus/submodules/networkx && /cluster/home/hicke\ y/genomes/progressiveCactus/python/bin/python setup.py install running install running build running build_py running install_lib creating /cluster/home/hickey/genomes/progressiveCactus/python/lib/python2.7/site-packages/networkx (etc)

From your log, I think it's failing on the second step, installing networkx in the virtual python:

. ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install

Note that ${virPyEnv} is progressiveCactus/environment, and made by a combination of submodules/Makefile and virtualenv.py.

This is the environment that's loaded to switch to the cactus python, and would be culprit if somehow your pythonhome is getting mixed up. I think maybe trying to reproduce with commands outside the Makefile might be the way to go... Wish I could reproduce here to help more.

On Mon, Oct 12, 2015 at 5:05 PM, rsharris notifications@github.com wrote:

No luck.

Reponse from those commands: Previous HEAD position was 239d5a9... Merge branch 'release/1.8.2' Branch master set up to track remote branch master from origin. Switched to a new branch 'master' baker:virtualenv$ git pull remote: Counting objects: 8, done. remote: Total 8 (delta 3), reused 3 (delta 3), pack-reused 5 Unpacking objects: 100% (8/8), done. From git://github.com/pypa/virtualenv 1076aff..cba89a1 develop -> origin/develop Already up-to-date.

And then a make (after cd back to the progressiveCactus directory) give the same output as before, except that it now reports overwriting the progressiveCactus site.py: Overwriting ...path.../progressiveCactus/python/Python-2.7.10/Lib/site.py with new content The rest of the output from make is identical to before.

As I noted earlier, firing up python from the shell the _sysconfigdata import is OK. This however is using the python I built and installed, not the one inside progressiveCactus. The error log indicates that's the python the exceptions are occurring in. But then right after that something (I can't figure out what) appears to be reporting a problem with the progressiveCactus python. Maybe I'm misinterpreting that.

Earlier in the log it tells me it has installed "New python executable" and gives me the path to the progressiveCactus python. Is the problem that the subsequent steps (the ones that fail) are supposed to be using the progressiveCactus python but for some reason are using the wrong one? --OR-- is the message about $PYTHONHOME and the prefix related to the progressiveCactus python instead of the python I would normally be running?

Thanks, Bob H

On Oct 12, 2015, at 4:40 PM, Glenn Hickey wrote:

I see no reason not to update the virtualenv submodule in progressive cactus.

Can you please verify that doing so would fix your issue though?

should be as simple as

cd submodules/virtualenv git checkout master git pull

thanks! -Glenn

On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com wrote:

Turns out we aren't using Ubuntu. But looking at the first link, second answer, I'm guessing my virtualenv is too old. It is version 1.5.something, and this site advises not using anything before 1.9: https://virtualenv.pypa.io/en/latest/installation.html

I think I'll have to get an admin to do the virtualenv install. Before I go down that path, do you think that's a good path to head down? Is there a (known) expectation by the cactus package of virtualenv version (I realize no one else may have run into such a problem).

Thanks, Bob H

On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote:

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6,

but

perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules

' rm -f /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment

python /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py

/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python

New python executable in /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 548, in main() File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 520, in get_config_var return get_config_vars().get(name) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python

is not functioning ERROR: It thinks sys.prefix is u'/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules '

(should be u'/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python

') ERROR: virtualenv is not compatible with this system or executable make[1]: *\ [virtPyRule] Error 100 make[1]: Leaving directory/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules

' make: *\ [all] Error 2

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub < https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147522813 .

— Reply to this email directly or view it on GitHub.

[rsharris]

Well, I guess I have figured one thing out, maybe. When the cactus version of python is run, I get exceptions reported from my normal python. If I suppress site.py with the -S option, that changes.

Specifically, if I run the cactus version of python from the shell, like this, it fails to get to the within-python user prompt.

virtPyDir=/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python virtPyEnv=/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/activate virtPy=/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python myEnv=/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment ${virtPy}

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in boot imp.load_module('site',stream,path,descr) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 548, in main() File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 520, in get_config_var return get_config_vars().get(name) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata

This is the meat of the error I get when I do make. Looking at the exception list, I thought maybe this should not be using my site.py.

So, running the cactus version of python from the shell, like this, it will get to the user prompt. Unfortunately it still won't do the import. Also odd is the fact that it gets the prefix for my normal python build. Where is it getting that from? (it was built into my python, but I have not set PYTHONHOME).

${virtPy} -S

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2

from _sysconfigdata import build_time_vars Traceback (most recent call last): File "", line 1, in ImportError: No module named _sysconfigdata

from sys import prefix

prefix '/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64'

I'd like to try adding the -S switch to what the makefile is doing, but I can't see where I would do that. The first thing in the virtPyRule target is python ${PWD}/virtualenv/virtualenv.py ${virtPyDir}, and the failure occurs somewhere in there. I'm assuming this is just a standard virtualenv.py, not something written especially for cactus, right?

I have figured out that the other massages I get from doing make come from virtualenv.py, in the install_python function. Adding some debug prints to the front of that function, it looks like it is being given the correct paths.

home_dir = "/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python" lib_dir = "/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/lib/python2.7" inc_dir = "/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/include/python2.7" bin_dir = "/afs/bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin"

With some more digging I guess I can try suppressing site.py and see if that changes anything. It may just be a red herring. But I can't think of anything else to try.

Thinking out loud ... what would it take to just try running cactus without using the virtualenv stuff? Will I just run into different problems?

Bob H

On Oct 12, 2015, at 5:28 PM, Glenn Hickey wrote:

I'm not sure. For what it's worth, these are the commands in the Makefile for this part:

python $(PWD)/virtualenv/virtualenv.py ${virtPyDir} . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install . ${virtPyEnv} && cd $(PWD)/psutil/ && ${virtPy} setup.py install . ${virtPyEnv} && cd $(PWD)/biopython/ && echo y | ${virtPy} setup.py install | cat cp ${virtPyEnv} ${myEnv} echo export PYTHONPATH=$(PWD):\$$PYTHONPATH >> ${myEnv} echo export PATH=${virtPyDir}/bin:\$$PATH >> ${myEnv}

On our server the (relevant) part of the output looks like:

rm -f /cluster/home/hickey/genomes/progressiveCactus/submodules/../environment python /cluster/home/hickey/genomes/progressiveCactus/submodules/virtualenv/virtualenv.py /cluster/home/hickey/genomes/progressiveCactus/python New python executable in /cluster/home/hickey/genomes/progressiveCactus/python/bin/python Installing setuptools............done. Installing pip...............done. . /cluster/home/hickey/genomes/progressiveCactus/python/bin/activate && cd /cluster/home/hickey/genomes/progressiveCactus/submodules/networkx && /cluster/home/hicke\ y/genomes/progressiveCactus/python/bin/python setup.py install running install running build running build_py running install_lib creating /cluster/home/hickey/genomes/progressiveCactus/python/lib/python2.7/site-packages/networkx (etc)

From your log, I think it's failing on the second step, installing networkx in the virtual python:

. ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install

Note that ${virPyEnv} is progressiveCactus/environment, and made by a combination of submodules/Makefile and virtualenv.py.

This is the environment that's loaded to switch to the cactus python, and would be culprit if somehow your pythonhome is getting mixed up. I think maybe trying to reproduce with commands outside the Makefile might be the way to go... Wish I could reproduce here to help more.

On Mon, Oct 12, 2015 at 5:05 PM, rsharris notifications@github.com wrote:

No luck.

Reponse from those commands: Previous HEAD position was 239d5a9... Merge branch 'release/1.8.2' Branch master set up to track remote branch master from origin. Switched to a new branch 'master' baker:virtualenv$ git pull remote: Counting objects: 8, done. remote: Total 8 (delta 3), reused 3 (delta 3), pack-reused 5 Unpacking objects: 100% (8/8), done. From git://github.com/pypa/virtualenv 1076aff..cba89a1 develop -> origin/develop Already up-to-date.

And then a make (after cd back to the progressiveCactus directory) give the same output as before, except that it now reports overwriting the progressiveCactus site.py: Overwriting ...path.../progressiveCactus/python/Python-2.7.10/Lib/site.py with new content The rest of the output from make is identical to before.

As I noted earlier, firing up python from the shell the _sysconfigdata import is OK. This however is using the python I built and installed, not the one inside progressiveCactus. The error log indicates that's the python the exceptions are occurring in. But then right after that something (I can't figure out what) appears to be reporting a problem with the progressiveCactus python. Maybe I'm misinterpreting that.

Earlier in the log it tells me it has installed "New python executable" and gives me the path to the progressiveCactus python. Is the problem that the subsequent steps (the ones that fail) are supposed to be using the progressiveCactus python but for some reason are using the wrong one? --OR-- is the message about $PYTHONHOME and the prefix related to the progressiveCactus python instead of the python I would normally be running?

Thanks, Bob H

On Oct 12, 2015, at 4:40 PM, Glenn Hickey wrote:

I see no reason not to update the virtualenv submodule in progressive cactus.

Can you please verify that doing so would fix your issue though?

should be as simple as

cd submodules/virtualenv git checkout master git pull

thanks! -Glenn

On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com wrote:

Turns out we aren't using Ubuntu. But looking at the first link, second answer, I'm guessing my virtualenv is too old. It is version 1.5.something, and this site advises not using anything before 1.9: https://virtualenv.pypa.io/en/latest/installation.html

I think I'll have to get an admin to do the virtualenv install. Before I go down that path, do you think that's a good path to head down? Is there a (known) expectation by the cactus package of virtualenv version (I realize no one else may have run into such a problem).

Thanks, Bob H

On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote:

Hi Bob,

Hard for me to say anything without being able to reproduce, but googling gives:

http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u

which includes a possible workaround. It also refers to this report for a bug in Ubuntu:

https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466

maybe this is related?

cheers -Glenn

On Mon, Oct 12, 2015 at 3:52 PM, rsharris notifications@github.com wrote:

(previously posted this as an add on to issue #41 https://github.com/glennhickey/progressiveCactus/issues/41, Oct/6,

but

perhaps posts to closed issues go unnoticed)

I've finally gotten back to this. I rebuilt my python and correctly set the --prefix option when building that (also upgrading from 2.7.4 to 2.7.10). This has gotten me past the problem in my Sep/8 post, but the "make" quickly runs into another one. I have attached the output from make, as well as appending it below.

I believe the PYTHONHOME stuff is a red herring. If I follow the suggestion in line 7 to set PYTHONHOME (to /home/me/python/linux-x86_64), that message goes away but the rest of the output is (I believe) the same. So for the run that produced this log, PYTHONHOME isn't set.

One source of frustration is that I cannot find the modules that are outputting any of the informational messages here (other than the modules identified in the traceback). If I could find the module telling me what it thinks about sys.prefix, I might be able to figure out ... something. I have tried searching the cactus directories for pieces of that error text, but to no avail. From the command line python shell, sys.prefix is /home/me/python/linux-x86_64.

Further, the failed attempt to import _sysconfigdata is hard to decipher. If I fire up python from the command line, there is no problem with that import: $ python Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 Type "help", "copyright", "credits" or "license" for more information.

from _sysconfigdata import build_time_vars len(build_time_vars) 532 So this would suggest something is mucking with where python is looking for that module. Which is no surprise for anything involving virtualenv. But as I am unable to locate the mucking modules I'm unable to make any guesses about how things have been mucked up.

Here are the commands I did, starting from a fresh pull from github Oct/5: cd /home/me/cactus git clone git://github.com/glennhickey/progressiveCactus.git cd progressiveCactus git pull git submodule update --init make

Thanks for any help. Bob H

==== output from make ==== (I think formatting with much this up)

cd submodules && make all make[1]: Entering directory /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules

' rm -f /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment

python /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py

/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python

New python executable in /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 548, in main() File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py ",

line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 520, in get_config_var return get_config_vars().get(name) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/

bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py ",

line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata ERROR: The executable /afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python

is not functioning ERROR: It thinks sys.prefix is u'/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules '

(should be u'/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python

') ERROR: virtualenv is not compatible with this system or executable make[1]: *\ [virtPyRule] Error 100 make[1]: Leaving directory/afs/

bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules

' make: *\ [all] Error 2

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub < https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627

.

— Reply to this email directly or view it on GitHub.

— Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147522813 .

— Reply to this email directly or view it on GitHub.

[glennhickey]

Yeah, this has already gone way past my level of expertise with virtualenv / python environments. Completely bizarre that it only seems to happen to you.

No reason indeed why you can't just bypass virtualenv. I think it should be as simple as cutting out the "virtPyRule" rule from submodules/Makefile (you will have to make sure that networkx, psutil and biopython are installed yourself). The environment file should still be written, I think, and the rest of the submodules should still work as designed as they have nothing to do witih python.

On Tue, Oct 13, 2015 at 1:08 PM, rsharris notifications@github.com wrote:

Well, I guess I have figured one thing out, maybe. When the cactus version of python is run, I get exceptions reported from my normal python. If I suppress site.py with the -S option, that changes.

Specifically, if I run the cactus version of python from the shell, like this, it fails to get to the within-python user prompt.

virtPyDir=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python virtPyEnv=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/activate virtPy=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python myEnv=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment ${virtPy}

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 548, in main() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 520, in get_config_var return get_config_vars().get(name) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata This is the meat of the error I get when I do make. Looking at the exception list, I thought maybe this should not be using my site.py. So, running the cactus version of python from the shell, like this, it will get to the user prompt. Unfortunately it still won't do the import. Also odd is the fact that it gets the prefix for my normal python build. Where is it getting that from? (it was built into _my_ python, but I have not set PYTHONHOME). ${virtPy} -S Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 > > > from _sysconfigdata import build_time_vars > > > Traceback (most recent call last): > > > File "", line 1, in > > > ImportError: No module named _sysconfigdata from sys import prefix > > > prefix > > > '/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64 > > > ' I'd like to try adding the -S switch to what the makefile is doing, but I can't see where I would do that. The first thing in the virtPyRule target is python ${PWD}/virtualenv/virtualenv.py ${virtPyDir}, and the failure occurs somewhere in there. I'm assuming this is just a standard virtualenv.py, not something written especially for cactus, right? I have figured out that the other massages I get from doing make come from virtualenv.py, in the install_python function. Adding some debug prints to the front of that function, it looks like it _is_ being given the correct paths. home_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python " lib_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/lib/python2.7 " inc_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/include/python2.7 " bin_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin " With some more digging I guess I can try suppressing site.py and see if that changes anything. It may just be a red herring. But I can't think of anything else to try. Thinking out loud ... what would it take to just try running cactus without using the virtualenv stuff? Will I just run into different problems? Bob H On Oct 12, 2015, at 5:28 PM, Glenn Hickey wrote: > I'm not sure. For what it's worth, these are the commands in the Makefile > for this part: > > python $(PWD)/virtualenv/virtualenv.py ${virtPyDir} > . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install > . ${virtPyEnv} && cd $(PWD)/psutil/ && ${virtPy} setup.py install > . ${virtPyEnv} && cd $(PWD)/biopython/ && echo y | ${virtPy} setup.py > install | cat > cp ${virtPyEnv} ${myEnv} > echo export PYTHONPATH=$(PWD):\$$PYTHONPATH >> ${myEnv} > echo export PATH=${virtPyDir}/bin:\$$PATH >> ${myEnv} > > On our server the (relevant) part of the output looks like: > > rm -f > /cluster/home/hickey/genomes/progressiveCactus/submodules/../environment > python > > /cluster/home/hickey/genomes/progressiveCactus/submodules/virtualenv/virtualenv.py > /cluster/home/hickey/genomes/progressiveCactus/python > New python executable in > /cluster/home/hickey/genomes/progressiveCactus/python/bin/python > Installing setuptools............done. > Installing pip...............done. > . /cluster/home/hickey/genomes/progressiveCactus/python/bin/activate && > cd > /cluster/home/hickey/genomes/progressiveCactus/submodules/networkx && > /cluster/home/hicke\ > y/genomes/progressiveCactus/python/bin/python setup.py install > running install > running build > running build_py > running install_lib > creating > > /cluster/home/hickey/genomes/progressiveCactus/python/lib/python2.7/site-packages/networkx > (etc) > > From your log, I think it's failing on the second step, installing > networkx > in the virtual python: > > . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install > > Note that ${virPyEnv} is progressiveCactus/environment, and made by a > combination of submodules/Makefile and virtualenv.py. > > This is the environment that's loaded to switch to the cactus python, and > would be culprit if somehow your pythonhome is getting mixed up. I think > maybe trying to reproduce with commands outside the Makefile might be the > way to go... Wish I could reproduce here to help more. > > On Mon, Oct 12, 2015 at 5:05 PM, rsharris notifications@github.com > wrote: > > > No luck. > > > > Reponse from those commands: > > Previous HEAD position was 239d5a9... Merge branch 'release/1.8.2' > > Branch master set up to track remote branch master from origin. > > Switched to a new branch 'master' > > baker:virtualenv$ git pull > > remote: Counting objects: 8, done. > > remote: Total 8 (delta 3), reused 3 (delta 3), pack-reused 5 > > Unpacking objects: 100% (8/8), done. > > From git://github.com/pypa/virtualenv > > 1076aff..cba89a1 develop -> origin/develop > > Already up-to-date. > > > > And then a make (after cd back to the progressiveCactus directory) give > > the same output as before, except that it now reports overwriting the > > progressiveCactus site.py: > > Overwriting > > ...path.../progressiveCactus/python/Python-2.7.10/Lib/site.py > > with new content > > The rest of the output from make is identical to before. > > > > As I noted earlier, firing up python from the shell the _sysconfigdata > > import is OK. This however is using the python I built and installed, > > not > > the one inside progressiveCactus. The error log indicates that's the > > python > > the exceptions are occurring in. But then right after that _something_ > > (I > > can't figure out what) appears to be reporting a problem with the > > progressiveCactus python. Maybe I'm misinterpreting that. > > > > Earlier in the log it tells me it has installed "New python executable" > > and gives me the path to the progressiveCactus python. Is the problem > > that > > the subsequent steps (the ones that fail) are supposed to be using the > > progressiveCactus python but for some reason are using the wrong one? > > --OR-- > > is the message about $PYTHONHOME and the prefix related to the > > progressiveCactus python instead of the python I would normally be > > running? > > > > Thanks, > > Bob H > > > > On Oct 12, 2015, at 4:40 PM, Glenn Hickey wrote: > > > > > I see no reason not to update the virtualenv submodule in progressive > > > cactus. > > > > > > Can you please verify that doing so would fix your issue though? > > > > > > should be as simple as > > > > > > cd submodules/virtualenv > > > git checkout master > > > git pull > > > > > > thanks! > > > -Glenn > > > > > > On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com > > > wrote: > > > > > > > Turns out we aren't using Ubuntu. But looking at the first link, > > > > second > > > > answer, I'm guessing my virtualenv is too old. It is version > > > > 1.5.something, > > > > and this site advises not using anything before 1.9: > > > > https://virtualenv.pypa.io/en/latest/installation.html > > > > > > > > I _think_ I'll have to get an admin to do the virtualenv install. > > > > Before I > > > > go down that path, do you think that's a good path to head down? Is > > > > there a > > > > (known) expectation by the cactus package of virtualenv version (I > > > > realize > > > > no one else may have run into such a problem). > > > > > > > > Thanks, > > > > Bob H > > > > > > > > On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote: > > > > > > > > > Hi Bob, > > > > > > > > > > Hard for me to say anything without being able to reproduce, but > > > > > googling > > > > > gives: > > > > http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u > > > > > > > which includes a possible workaround. It also refers to this > > > > > report > > > > > for a > > > > > bug in Ubuntu: > > > > > > > > > > https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466 > > > > > > > > > > maybe this is related? > > > > > > > > > > cheers > > > > > -Glenn > > > > > > > > > > On Mon, Oct 12, 2015 at 3:52 PM, rsharris < > > > > > notifications@github.com> > > > > > wrote: > > > > > > > > > > > (previously posted this as an add on to issue #41 > > > > > > https://github.com/glennhickey/progressiveCactus/issues/41, > > > > Oct/6, > > > > > > but > > > > > > > > > > perhaps posts to closed issues go unnoticed) > > > > > > > > > > > > I've finally gotten back to this. I rebuilt my python and > > > > > > correctly set > > > > > > the --prefix option when building that (also upgrading from > > > > > > 2.7.4 > > > > > > to > > > > > > 2.7.10). This has gotten me past the problem in my Sep/8 post, > > > > > > but > > > > > > the > > > > > > "make" quickly runs into another one. I have attached the > > > > > > output > > > > > > from > > > > > > make, > > > > > > as well as appending it below. > > > > > > > > > > > > I believe the PYTHONHOME stuff is a red herring. If I follow > > > > > > the > > > > > > suggestion in line 7 to set PYTHONHOME (to > > > > > > /home/me/python/linux-x86_64), > > > > > > that message goes away but the rest of the output is (I > > > > > > believe) > > > > > > the > > > > > > same. > > > > > > So for the run that produced this log, PYTHONHOME isn't set. > > > > > > > > > > > > One source of frustration is that I cannot find the modules > > > > > > that > > > > > > are > > > > > > outputting any of the informational messages here (other than > > > > > > the > > > > > > modules > > > > > > identified in the traceback). If I could find the module > > > > > > telling me > > > > > > what it > > > > > > thinks about sys.prefix, I might be able to figure out ... > > > > > > something. I > > > > > > have tried searching the cactus directories for pieces of that > > > > > > error > > > > > > text, > > > > > > but to no avail. From the command line python shell, > > > > > > sys.prefix is > > > > > > /home/me/python/linux-x86_64. > > > > > > > > > > > > Further, the failed attempt to import _sysconfigdata is hard to > > > > > > decipher. > > > > > > If I fire up python from the command line, there is no problem > > > > > > with > > > > > > that > > > > > > import: > > > > > > $ python > > > > > > Python 2.7.10 (default, Oct 5 2015, 16:35:54) > > > > > > [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 > > > > > > Type "help", "copyright", "credits" or "license" for more > > > > > > information. > > > > > > > > > > > > from _sysconfigdata import build_time_vars > > > > > > len(build_time_vars) > > > > > > 532 > > > > > > So this would suggest something is mucking with where python is > > > > > > looking > > > > > > for that module. Which is no surprise for anything involving > > > > > > virtualenv. > > > > > > But as I am unable to locate the mucking modules I'm unable to > > > > > > make any > > > > > > guesses about how things have been mucked up. > > > > > > > > > > > > Here are the commands I did, starting from a fresh pull from > > > > > > github > > > > > > Oct/5: > > > > > > cd /home/me/cactus > > > > > > git clone git://github.com/glennhickey/progressiveCactus.git > > > > > > cd progressiveCactus > > > > > > git pull > > > > > > git submodule update --init > > > > > > make > > > > > > > > > > > > Thanks for any help. > > > > > > Bob H > > > > > > > > > > > > ==== output from make ==== (I think formatting with much this > > > > > > up) > > > > > > > > > > > > cd submodules && make all > > > > > > make[1]: Entering directory /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules > > > > > > > > ' > > > > > > rm -f /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment > > > > > > > > python /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py > > > > > > > > /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python > > > > > > > > New python executable in /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python > > > > > > > > Could not find platform dependent libraries > > > > > > Consider setting $PYTHONHOME to [:] > > > > > > Traceback (most recent call last): > > > > > > File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", > > > > > > line > > > > > > 75, > > > > > > in > > > > > > > > > > > > __boot() > > > > > > File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", > > > > > > line > > > > > > 35, > > > > > > in > > > > > > __boot > > > > > > imp.load_module('site',stream,path,descr) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 548, in > > > > > > main() > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 530, in main > > > > > > known_paths = addusersitepackages(known_paths) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 266, in addusersitepackages > > > > > > user_site = getusersitepackages() > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 241, in getusersitepackages > > > > > > user_base = getuserbase() # this will also set USER_BASE > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 231, in getuserbase > > > > > > USER_BASE = get_config_var('userbase') > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py > > > > > > ", > > > > > > > > > > line 520, in get_config_var > > > > > > return get_config_vars().get(name) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py > > > > > > ", > > > > > > > > > > line 472, in get_config_vars > > > > > > _init_posix(_CONFIG_VARS) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py > > > > > > ", > > > > > > > > > > line 356, in _init_posix > > > > > > from _sysconfigdata import build_time_vars > > > > > > ImportError: No module named _sysconfigdata > > > > > > ERROR: The executable /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python > > > > > > > > is not functioning > > > > > > ERROR: It thinks sys.prefix is u'/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules > > > > > > ' > > > > > > > > > > (should be u'/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python > > > > > > > > ') > > > > > > ERROR: virtualenv is not compatible with this system or > > > > > > executable > > > > > > make[1]: **\* [virtPyRule] Error 100 > > > > > > make[1]: Leaving directory/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules > > > > > > > > ' > > > > > > make: **\* [all] Error 2 > > > > > > > > > > > > — > > > > > > Reply to this email directly or view it on GitHub > > > > > > https://github.com/glennhickey/progressiveCactus/issues/46. > > > > > > > > > > > > — > > > > > > Reply to this email directly or view it on GitHub. > > > > > > > > — > > > > Reply to this email directly or view it on GitHub > > > > < > > > > https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627 > > > > > > . > > > > > > > > — > > > > Reply to this email directly or view it on GitHub. > > > > — > > Reply to this email directly or view it on GitHub > > < > > https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147522813 > > > > . > > > > — > > Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147780145 .

[rsharris]

Yeah, it's probably some incompatibility between my python setup and virtualenv. I found a lot of similar reports online, but none of the answers appeared to apply to me. My best guess is that something I set up to get some package working five years ago is no longer kosher (site packages, pydistutil, or something like that). But I'm not willing to risk wiping out all that stuff for fear it will break something I use frequently.

Thanks for your help, Bob H

On Oct 14, 2015, at 1:33 AM, Glenn Hickey notifications@github.com wrote:

Yeah, this has already gone way past my level of expertise with virtualenv / python environments. Completely bizarre that it only seems to happen to you.

No reason indeed why you can't just bypass virtualenv. I think it should be as simple as cutting out the "virtPyRule" rule from submodules/Makefile (you will have to make sure that networkx, psutil and biopython are installed yourself). The environment file should still be written, I think, and the rest of the submodules should still work as designed as they have nothing to do witih python.

On Tue, Oct 13, 2015 at 1:08 PM, rsharris notifications@github.com wrote:

Well, I guess I have figured one thing out, maybe. When the cactus version of python is run, I get exceptions reported from my normal python. If I suppress site.py with the -S option, that changes.

Specifically, if I run the cactus version of python from the shell, like this, it fails to get to the within-python user prompt.

virtPyDir=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python virtPyEnv=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/activate virtPy=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python myEnv=/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment ${virtPy}

Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Traceback (most recent call last): File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 75, in

__boot() File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", line 35, in __boot imp.load_module('site',stream,path,descr) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 548, in main() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 530, in main known_paths = addusersitepackages(known_paths) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 266, in addusersitepackages user_site = getusersitepackages() File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 241, in getusersitepackages user_base = getuserbase() # this will also set USER_BASE File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py", line 231, in getuserbase USER_BASE = get_config_var('userbase') File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 520, in get_config_var return get_config_vars().get(name) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 472, in get_config_vars _init_posix(_CONFIG_VARS) File "/afs/ bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py", line 356, in _init_posix from _sysconfigdata import build_time_vars ImportError: No module named _sysconfigdata This is the meat of the error I get when I do make. Looking at the exception list, I thought maybe this should not be using my site.py. So, running the cactus version of python from the shell, like this, it will get to the user prompt. Unfortunately it still won't do the import. Also odd is the fact that it gets the prefix for my normal python build. Where is it getting that from? (it was built into _my_ python, but I have not set PYTHONHOME). ${virtPy} -S Could not find platform dependent libraries Consider setting $PYTHONHOME to [:] Python 2.7.10 (default, Oct 5 2015, 16:35:54) [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 > > > from _sysconfigdata import build_time_vars > > > Traceback (most recent call last): > > > File "", line 1, in > > > ImportError: No module named _sysconfigdata from sys import prefix > > > prefix > > > '/afs/bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64 > > > ' I'd like to try adding the -S switch to what the makefile is doing, but I can't see where I would do that. The first thing in the virtPyRule target is python ${PWD}/virtualenv/virtualenv.py ${virtPyDir}, and the failure occurs somewhere in there. I'm assuming this is just a standard virtualenv.py, not something written especially for cactus, right? I have figured out that the other massages I get from doing make come from virtualenv.py, in the install_python function. Adding some debug prints to the front of that function, it looks like it _is_ being given the correct paths. home_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python " lib_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/lib/python2.7 " inc_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/include/python2.7 " bin_dir = "/afs/ bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin " With some more digging I guess I can try suppressing site.py and see if that changes anything. It may just be a red herring. But I can't think of anything else to try. Thinking out loud ... what would it take to just try running cactus without using the virtualenv stuff? Will I just run into different problems? Bob H On Oct 12, 2015, at 5:28 PM, Glenn Hickey wrote: > I'm not sure. For what it's worth, these are the commands in the Makefile > for this part: > > python $(PWD)/virtualenv/virtualenv.py ${virtPyDir} > . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install > . ${virtPyEnv} && cd $(PWD)/psutil/ && ${virtPy} setup.py install > . ${virtPyEnv} && cd $(PWD)/biopython/ && echo y | ${virtPy} setup.py > install | cat > cp ${virtPyEnv} ${myEnv} > echo export PYTHONPATH=$(PWD):\$$PYTHONPATH >> ${myEnv} > echo export PATH=${virtPyDir}/bin:\$$PATH >> ${myEnv} > > On our server the (relevant) part of the output looks like: > > rm -f > /cluster/home/hickey/genomes/progressiveCactus/submodules/../environment > python > > /cluster/home/hickey/genomes/progressiveCactus/submodules/virtualenv/virtualenv.py > /cluster/home/hickey/genomes/progressiveCactus/python > New python executable in > /cluster/home/hickey/genomes/progressiveCactus/python/bin/python > Installing setuptools............done. > Installing pip...............done. > . /cluster/home/hickey/genomes/progressiveCactus/python/bin/activate && > cd > /cluster/home/hickey/genomes/progressiveCactus/submodules/networkx && > /cluster/home/hicke\ > y/genomes/progressiveCactus/python/bin/python setup.py install > running install > running build > running build_py > running install_lib > creating > > /cluster/home/hickey/genomes/progressiveCactus/python/lib/python2.7/site-packages/networkx > (etc) > > From your log, I think it's failing on the second step, installing > networkx > in the virtual python: > > . ${virtPyEnv} && cd $(PWD)/networkx && ${virtPy} setup.py install > > Note that ${virPyEnv} is progressiveCactus/environment, and made by a > combination of submodules/Makefile and virtualenv.py. > > This is the environment that's loaded to switch to the cactus python, and > would be culprit if somehow your pythonhome is getting mixed up. I think > maybe trying to reproduce with commands outside the Makefile might be the > way to go... Wish I could reproduce here to help more. > > On Mon, Oct 12, 2015 at 5:05 PM, rsharris notifications@github.com > wrote: > > > No luck. > > > > Reponse from those commands: > > Previous HEAD position was 239d5a9... Merge branch 'release/1.8.2' > > Branch master set up to track remote branch master from origin. > > Switched to a new branch 'master' > > baker:virtualenv$ git pull > > remote: Counting objects: 8, done. > > remote: Total 8 (delta 3), reused 3 (delta 3), pack-reused 5 > > Unpacking objects: 100% (8/8), done. > > From git://github.com/pypa/virtualenv > > 1076aff..cba89a1 develop -> origin/develop > > Already up-to-date. > > > > And then a make (after cd back to the progressiveCactus directory) give > > the same output as before, except that it now reports overwriting the > > progressiveCactus site.py: > > Overwriting > > ...path.../progressiveCactus/python/Python-2.7.10/Lib/site.py > > with new content > > The rest of the output from make is identical to before. > > > > As I noted earlier, firing up python from the shell the _sysconfigdata > > import is OK. This however is using the python I built and installed, > > not > > the one inside progressiveCactus. The error log indicates that's the > > python > > the exceptions are occurring in. But then right after that _something_ > > (I > > can't figure out what) appears to be reporting a problem with the > > progressiveCactus python. Maybe I'm misinterpreting that. > > > > Earlier in the log it tells me it has installed "New python executable" > > and gives me the path to the progressiveCactus python. Is the problem > > that > > the subsequent steps (the ones that fail) are supposed to be using the > > progressiveCactus python but for some reason are using the wrong one? > > --OR-- > > is the message about $PYTHONHOME and the prefix related to the > > progressiveCactus python instead of the python I would normally be > > running? > > > > Thanks, > > Bob H > > > > On Oct 12, 2015, at 4:40 PM, Glenn Hickey wrote: > > > > > I see no reason not to update the virtualenv submodule in progressive > > > cactus. > > > > > > Can you please verify that doing so would fix your issue though? > > > > > > should be as simple as > > > > > > cd submodules/virtualenv > > > git checkout master > > > git pull > > > > > > thanks! > > > -Glenn > > > > > > On Mon, Oct 12, 2015 at 4:25 PM, rsharris notifications@github.com > > > wrote: > > > > > > > Turns out we aren't using Ubuntu. But looking at the first link, > > > > second > > > > answer, I'm guessing my virtualenv is too old. It is version > > > > 1.5.something, > > > > and this site advises not using anything before 1.9: > > > > https://virtualenv.pypa.io/en/latest/installation.html > > > > > > > > I _think_ I'll have to get an admin to do the virtualenv install. > > > > Before I > > > > go down that path, do you think that's a good path to head down? Is > > > > there a > > > > (known) expectation by the cactus package of virtualenv version (I > > > > realize > > > > no one else may have run into such a problem). > > > > > > > > Thanks, > > > > Bob H > > > > > > > > On Oct 12, 2015, at 3:57 PM, Glenn Hickey wrote: > > > > > > > > > Hi Bob, > > > > > > > > > > Hard for me to say anything without being able to reproduce, but > > > > > googling > > > > > gives: > > > > http://stackoverflow.com/questions/15608236/eclipse-and-google-app-engine-importerror-no-module-named-sysconfigdata-nd-u > > > > > > > which includes a possible workaround. It also refers to this > > > > > report > > > > > for a > > > > > bug in Ubuntu: > > > > > > > > > > https://bugs.launchpad.net/ubuntu/+source/python2.7/+bug/1115466 > > > > > > > > > > maybe this is related? > > > > > > > > > > cheers > > > > > -Glenn > > > > > > > > > > On Mon, Oct 12, 2015 at 3:52 PM, rsharris < > > > > > notifications@github.com> > > > > > wrote: > > > > > > > > > > > (previously posted this as an add on to issue #41 > > > > > > https://github.com/glennhickey/progressiveCactus/issues/41, > > > > Oct/6, > > > > > > but > > > > > > > > > > perhaps posts to closed issues go unnoticed) > > > > > > > > > > > > I've finally gotten back to this. I rebuilt my python and > > > > > > correctly set > > > > > > the --prefix option when building that (also upgrading from > > > > > > 2.7.4 > > > > > > to > > > > > > 2.7.10). This has gotten me past the problem in my Sep/8 post, > > > > > > but > > > > > > the > > > > > > "make" quickly runs into another one. I have attached the > > > > > > output > > > > > > from > > > > > > make, > > > > > > as well as appending it below. > > > > > > > > > > > > I believe the PYTHONHOME stuff is a red herring. If I follow > > > > > > the > > > > > > suggestion in line 7 to set PYTHONHOME (to > > > > > > /home/me/python/linux-x86_64), > > > > > > that message goes away but the rest of the output is (I > > > > > > believe) > > > > > > the > > > > > > same. > > > > > > So for the run that produced this log, PYTHONHOME isn't set. > > > > > > > > > > > > One source of frustration is that I cannot find the modules > > > > > > that > > > > > > are > > > > > > outputting any of the informational messages here (other than > > > > > > the > > > > > > modules > > > > > > identified in the traceback). If I could find the module > > > > > > telling me > > > > > > what it > > > > > > thinks about sys.prefix, I might be able to figure out ... > > > > > > something. I > > > > > > have tried searching the cactus directories for pieces of that > > > > > > error > > > > > > text, > > > > > > but to no avail. From the command line python shell, > > > > > > sys.prefix is > > > > > > /home/me/python/linux-x86_64. > > > > > > > > > > > > Further, the failed attempt to import _sysconfigdata is hard to > > > > > > decipher. > > > > > > If I fire up python from the command line, there is no problem > > > > > > with > > > > > > that > > > > > > import: > > > > > > $ python > > > > > > Python 2.7.10 (default, Oct 5 2015, 16:35:54) > > > > > > [GCC 4.1.2 20080704 (Red Hat 4.1.2-52)] on linux2 > > > > > > Type "help", "copyright", "credits" or "license" for more > > > > > > information. > > > > > > > > > > > > from _sysconfigdata import build_time_vars > > > > > > len(build_time_vars) > > > > > > 532 > > > > > > So this would suggest something is mucking with where python is > > > > > > looking > > > > > > for that module. Which is no surprise for anything involving > > > > > > virtualenv. > > > > > > But as I am unable to locate the mucking modules I'm unable to > > > > > > make any > > > > > > guesses about how things have been mucked up. > > > > > > > > > > > > Here are the commands I did, starting from a fresh pull from > > > > > > github > > > > > > Oct/5: > > > > > > cd /home/me/cactus > > > > > > git clone git://github.com/glennhickey/progressiveCactus.git > > > > > > cd progressiveCactus > > > > > > git pull > > > > > > git submodule update --init > > > > > > make > > > > > > > > > > > > Thanks for any help. > > > > > > Bob H > > > > > > > > > > > > ==== output from make ==== (I think formatting with much this > > > > > > up) > > > > > > > > > > > > cd submodules && make all > > > > > > make[1]: Entering directory /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules > > > > > > > > ' > > > > > > rm -f /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/../environment > > > > > > > > python /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules/virtualenv/virtualenv.py > > > > > > > > /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python > > > > > > > > New python executable in /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python > > > > > > > > Could not find platform dependent libraries > > > > > > Consider setting $PYTHONHOME to [:] > > > > > > Traceback (most recent call last): > > > > > > File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", > > > > > > line > > > > > > 75, > > > > > > in > > > > > > > > > > > > __boot() > > > > > > File "/afs/bx.psu.edu/home/rsharris/py/lib/python/site.py", > > > > > > line > > > > > > 35, > > > > > > in > > > > > > __boot > > > > > > imp.load_module('site',stream,path,descr) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 548, in > > > > > > main() > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 530, in main > > > > > > known_paths = addusersitepackages(known_paths) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 266, in addusersitepackages > > > > > > user_site = getusersitepackages() > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 241, in getusersitepackages > > > > > > user_base = getuserbase() # this will also set USER_BASE > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/site.py > > > > > > ", > > > > > > > > > > line 231, in getuserbase > > > > > > USER_BASE = get_config_var('userbase') > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py > > > > > > ", > > > > > > > > > > line 520, in get_config_var > > > > > > return get_config_vars().get(name) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py > > > > > > ", > > > > > > > > > > line 472, in get_config_vars > > > > > > _init_posix(_CONFIG_VARS) > > > > > > File "/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/python/linux-x86_64/lib/python2.7/sysconfig.py > > > > > > ", > > > > > > > > > > line 356, in _init_posix > > > > > > from _sysconfigdata import build_time_vars > > > > > > ImportError: No module named _sysconfigdata > > > > > > ERROR: The executable /afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python/bin/python > > > > > > > > is not functioning > > > > > > ERROR: It thinks sys.prefix is u'/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules > > > > > > ' > > > > > > > > > > (should be u'/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/python > > > > > > > > ') > > > > > > ERROR: virtualenv is not compatible with this system or > > > > > > executable > > > > > > make[1]: **\* [virtPyRule] Error 100 > > > > > > make[1]: Leaving directory/afs/ > > > > bx.psu.edu/home/rsharris/development/third_party/cactus/progressiveCactus/submodules > > > > > > > > ' > > > > > > make: **\* [all] Error 2 > > > > > > > > > > > > — > > > > > > Reply to this email directly or view it on GitHub > > > > > > https://github.com/glennhickey/progressiveCactus/issues/46. > > > > > > > > > > > > — > > > > > > Reply to this email directly or view it on GitHub. > > > > > > > > — > > > > Reply to this email directly or view it on GitHub > > > > < > > > > https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147510627 > > > > > > . > > > > > > > > — > > > > Reply to this email directly or view it on GitHub. > > > > — > > Reply to this email directly or view it on GitHub > > < > > https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147522813 > > > > . > > > > — > > Reply to this email directly or view it on GitHub. — Reply to this email directly or view it on GitHub https://github.com/glennhickey/progressiveCactus/issues/46#issuecomment-147780145 . — Reply to this email directly or view it on GitHub.