search

radical-cybertools / radical.saga

A Light-Weight Access Layer for Distributed Computing Infrastructure and Reference Implementation of the SAGA Python Language Bindings.
http://radical-cybertools.github.io/saga-python/
Other
83 stars 34 forks source link

GSISSH not found #741

Closed iparask closed 4 years ago

iparask commented 4 years ago

I get this error when running an RP example:


During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/pilot/pmgr/launching/default.py", line 486, in work
    self._start_pilot_bulk(resource, schema, pilots)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/pilot/pmgr/launching/default.py", line 657, in _start_pilot_bulk
    fs.copy(tar_url, tar_rem, flags=rsfs.CREATE_PARENTS)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/namespace/directory.py", line 354, in copy
    if url_2: return self._adaptor.copy(url_1, url_2, flags, ttype=ttype)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/adaptors/cpi/decorators.py", line 62, in wrap_function
    return sync_function (self, *args, **kwargs)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/adaptors/shell/shell_file.py", line 553, in copy
    tgt.path, rec_flag)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 888, in stage_to_remote
    raise ptye.translate_exception (e)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 885, in stage_to_remote
    return self.run_copy_to (src, tgt, cp_flags)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 971, in run_copy_to
    self.cp_slave = self.factory.get_cp_slave (s_cmd, info, posix)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 435, in get_cp_slave
    self._initialize_pty (cp_slave, info, posix)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 420, in _initialize_pty
    raise ptye.translate_exception (e)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 266, in _initialize_pty
    n, match = pty_shell.find (prompt_patterns, delay)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 802, in find
    raise ptye.translate_exception (e, "(%s)" % data)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 799, in find
    data += self.read (timeout=_POLLDELAY)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 683, in read
    % (e, self.tail))
radical.saga.exceptions.NoSuccess: read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))

I realized that there are two / in the path:

>>> import radical.utils as ru
>>> ru.which('gsissh')
'/usr/local/globus-6/bin//gsissh'

This is on the Jetstream VM that has gsissh preinstalled

andre-merzky commented 4 years ago

hi @iparask : the above says During handling of the above exception, another exception occurred. What was that original exception?

iparask commented 4 years ago

Let me check.

iparask commented 4 years ago

From pmgr.0000.launching.0.log:

2019-10-30 10:57:11,514: pmgr.0000.launching.0: pmgr.0000.launching.0           : MainThread     : DEBUG   : cmd: cd /tmp/rp_agent_tar_dirwz2xto14 && tar zchf /tmp/rp_agent_tar_dirwz2xto14/rp.session.js-16-126.jetstream-cloud.org.iparask.018199.0003.pmgr.0000.launching.0.child.tgz *
2019-10-30 10:57:11,592: pmgr.0000.launching.0: pmgr.0000.launching.0           : MainThread     : DEBUG   : rs.file.Directory ('gsisftp://bridges.psc.xsede.org:2222/')
2019-10-30 10:57:12,701: pmgr.0000.launching.0: pmgr.0000.launching.0           : MainThread     : ERROR   : bulk launch failed
Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 603, in read
    buf = os.read (f, readsize)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/pilot/pmgr/launching/default.py", line 484, in work
    self._start_pilot_bulk(resource, schema, pilots)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/pilot/pmgr/launching/default.py", line 656, in _start_pilot_bulk
    fs.copy(tar_url, tar_rem, flags=rsfs.CREATE_PARENTS)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/namespace/directory.py", line 354, in copy
    if url_2: return self._adaptor.copy(url_1, url_2, flags, ttype=ttype)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/adaptors/cpi/decorators.py", line 62, in wrap_function
    return sync_function (self, *args, **kwargs)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/adaptors/shell/shell_file.py", line 553, in copy
    tgt.path, rec_flag)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 888, in stage_to_remote
    raise ptye.translate_exception (e)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 885, in stage_to_remote
    return self.run_copy_to (src, tgt, cp_flags)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 971, in run_copy_to
    self.cp_slave = self.factory.get_cp_slave (s_cmd, info, posix)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 435, in get_cp_slave
    self._initialize_pty (cp_slave, info, posix)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 420, in _initialize_pty
    raise ptye.translate_exception (e)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 266, in _initialize_pty
    n, match = pty_shell.find (prompt_patterns, delay)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 802, in find
    raise ptye.translate_exception (e, "(%s)" % data)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 799, in find
    data += self.read (timeout=_POLLDELAY)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 683, in read
    % (e, self.tail))
radical.saga.exceptions.NoSuccess: read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))
2019-10-30 10:57:12,702: pmgr.0000.launching.0: pmgr.0000.launching.0           : MainThread     : DEBUG   : advance bulk size: 1 [False, True]

and from radical.saga.pty:

2019-10-30 10:57:20,930: radical.saga.pty    : umgr.0000.staging.input.0       : MainThread     : DEBUG   : Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 603, in read
    buf = os.read (f, readsize)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 799, in find
    data += self.read (timeout=_POLLDELAY)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 683, in read
    % (e, self.tail))
radical.saga.exceptions.NoSuccess: read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))

2019-10-30 10:57:20,930: radical.saga.pty    : umgr.0000.staging.input.0       : MainThread     : ERROR   : read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))
Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 603, in read
    buf = os.read (f, readsize)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 266, in _initialize_pty
    n, match = pty_shell.find (prompt_patterns, delay)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 802, in find
    raise ptye.translate_exception (e, "(%s)" % data)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 799, in find
    data += self.read (timeout=_POLLDELAY)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 683, in read
    % (e, self.tail))
radical.saga.exceptions.NoSuccess: read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))
2019-10-30 10:57:20,931: radical.saga.pty    : umgr.0000.staging.input.0       : MainThread     : DEBUG   : Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 603, in read
    buf = os.read (f, readsize)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 266, in _initialize_pty
    n, match = pty_shell.find (prompt_patterns, delay)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 802, in find
    raise ptye.translate_exception (e, "(%s)" % data)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 799, in find
    data += self.read (timeout=_POLLDELAY)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 683, in read
    % (e, self.tail))
radical.saga.exceptions.NoSuccess: read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))

2019-10-30 10:57:20,932: radical.saga.pty    : umgr.0000.staging.input.0       : MainThread     : DEBUG   : Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 603, in read
    buf = os.read (f, readsize)
OSError: [Errno 5] Input/output error

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 885, in stage_to_remote
    return self.run_copy_to (src, tgt, cp_flags)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell.py", line 971, in run_copy_to
    self.cp_slave = self.factory.get_cp_slave (s_cmd, info, posix)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 435, in get_cp_slave
    self._initialize_pty (cp_slave, info, posix)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 420, in _initialize_pty
    raise ptye.translate_exception (e)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_shell_factory.py", line 266, in _initialize_pty
    n, match = pty_shell.find (prompt_patterns, delay)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 802, in find
    raise ptye.translate_exception (e, "(%s)" % data)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 799, in find
    data += self.read (timeout=_POLLDELAY)
  File "/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py", line 683, in read
    % (e, self.tail))
radical.saga.exceptions.NoSuccess: read from process failed '[Errno 5] Input/output error' : (exec: /usr/bin/gsissh: No such file or directory
Couldn't read packet: Connection reset by peer
) (/home/iparask/test_rp/lib/python3.5/site-packages/radical/saga/utils/pty_process.py +683 (read)  :  % (e, self.tail)))

2019-10-30 10:57:23,210: radical.saga.pty    : umgr.0000.staging.input.0       : MainThread     : DEBUG   : PTYProcess del  <radical.saga.utils.pty_process.PTYProcess object at 0x7fde1005e1d0>
andre-merzky commented 4 years ago

I am not sure what we can do if two gsissh executables are in the path, and one is not usable. Would you mind if I close this? The fix is to contact the sysadmin...

iparask commented 4 years ago

Let's close it.