agent startup error - what does it mean?

[andre-merzky]

On agent startup, I see in the error log:

merzky@thinkie:~ $ cat stderr-bj-91a6420a-5168-11e3-b13d-00231582da34-agent.txt
Traceback (most recent call last):
  File "/home/merzky/.bigjob/bigjob-bootstrap.py", line 1967, in <module>
    main()
  File "/home/merzky/.bigjob/bigjob-bootstrap.py", line 817, in main
    never_download=options.never_download)
  File "/home/merzky/.bigjob/bigjob-bootstrap.py", line 917, in create_environment
    search_dirs=search_dirs, never_download=never_download)
  File "/home/merzky/.bigjob/bigjob-bootstrap.py", line 610, in install_setuptools
    search_dirs=search_dirs, never_download=never_download)
  File "/home/merzky/.bigjob/bigjob-bootstrap.py", line 584, in _install_req
    cwd=cwd)
  File "/home/merzky/.bigjob/bigjob-bootstrap.py", line 885, in call_subprocess
    % (cmd_desc, proc.returncode))
OSError: Command /home/merzky/.bigjob/python/bin/python -c "#!python
\"\"\"Bootstra...sys.argv[1:])

" --always-copy -U setuptools failed with error code 1
/bin/sh: 1: aprun: not found
/bin/sh: 1: ibrun: not found
/bin/sh: 1: srun: not found
merzky@thinkie:~ $ python --version
Python 2.7.3

What does it mean? What can I do about it?

Thanks, Andre.

[melrom]

Andre - @drelu will definitely have a better answer than me, but as you already probably know, the bootstrap is failing.

My only question was - are you trying to bootstrap (i.e. is the target) a non-traditional resource?

If it is your localhost (and you need to get up and running before AL has a chance to look at this), put your BigJob executable in the virtualenv $HOME/.bigjob/python instead of $HOME/.bigjob. BigJob only tries to bootstrap itself onto resources where it does not detect a bigjob in $HOME/.bigjob/python. I don't know if this is entirely making sense. What I am trying to say is instead of:

virtualenv $HOME/.bigjob source $HOME/.bigjob/bin/activate

you say: virtualenv $HOME/.bigjob/python source $HOME/.bigjob/python/bin/activate

In the meantime, I will have a look at the bootstrap script. Also, if you're submitting to a different target host, then my previous comments are irrelevant. Also I am not trying to comment on the "correctness" of things, I am just providing a viable workaround for the localhost scenario.

[andre-merzky]

Melissa, thanks for the quick reply!

I indeed try on localhost at the moment -- thus a very traditional resource ;)

I am not sure I understood your workaround though, I'm afraid. I am nowhere doing source $HOME/.bigjob/bin/activate -- so not sure where to replace that. A virtualenv in $HOME/.bigjob/python seems to exist, and can be sourced -- but I am using a different VE for my stuff. So, do I need to source this one? Also, what do you mean with put your BigJob executable in the virtualenv $HOME/.bigjob/python instead of $HOME/.bigjob -- the only other thing in $HOME/.bigjob, besides python/, is bigjob-bootstrap.py -- is that what you mean?

Thanks!

Best, Andre.

[drelu]

Hi Andre, The bootstrap script 1. attempts to use BigJob in your PYTHONPATH, if not found then in 2. $HOME/.bigjob/python and if not found there it is attempting to download it via the bootstrap script, which seems to fail here:

python bigjob-bootstrap.py <$PATH>

In your system level Python, what version of setuptools, distribute and virtualenv do you have?

Thanks,

Andre

On Wed, Nov 20, 2013 at 1:59 AM, Andre Merzky notifications@github.comwrote:

Melissa, thanks for the quick reply!

I indeed try on localhost at the moment -- thus a very traditional resource ;)

I am not sure I understood your workaround though, I'm afraid. I am nowhere doing source $HOME/.bigjob/bin/activate -- so not sure where to replace that. A virtualenv in $HOME/.bigjob/python seems to exist, and can be sourced -- but I am using a different VE for my stuff. So, do I need to source this one? Also, what do you mean with put your BigJob executable in the virtualenv $HOME/.bigjob/python instead of $HOME/.bigjob-- the only other thing in $HOME/.bigjob, besides python/, is bigjob-bootstrap.py -- is that what you mean?

Thanks!

Best, Andre.

— Reply to this email directly or view it on GitHubhttps://github.com/saga-project/BigJob/issues/164#issuecomment-28867556 .

[andre-merzky]

Thanks! :)