NaturalHPC / cerulean

A Python 3 library for talking to HPC clusters and supercomputers
Apache License 2.0
7 stars 2 forks source link

Copy leaks paramiko exception #5

Open LourensVeen opened 5 years ago

LourensVeen commented 5 years ago

If the connection drops during a file copy operation, then a paramiko.ssh_exception.SSHConnectionError is raised. It would probably be better if you got an ordinary Python ConnectionError, or a Cerulean-specific exception type, as the fact that we use paramiko is really an implementation detail that should not affect the public API.

Backtrace:

  File "/usr/local/lib/python3.5/dist-packages/cerulean/copy_files.py", line 82, in copy
    copy_permissions, source_path, callback, 0, size)
  File "/usr/local/lib/python3.5/dist-packages/cerulean/copy_files.py", line 124, in _copy
    already_written, size)
  File "/usr/local/lib/python3.5/dist-packages/cerulean/copy_files.py", line 258, in _copy_dir
    permission, source_path.has_permission(permission))
  File "/usr/local/lib/python3.5/dist-packages/cerulean/path.py", line 516, in set_permission
    self.filesystem._set_permission(self.__path, permission, value)
  File "/usr/local/lib/python3.5/dist-packages/cerulean/sftp_file_system.py", line 283, in _set_permission
    self._chmod(lpath, mode)
  File "/usr/local/lib/python3.5/dist-packages/cerulean/sftp_file_system.py", line 288, in _chmod
    self.__sftp.chmod(str(lpath), mode)
  File "/usr/local/lib/python3.5/dist-packages/paramiko/sftp_client.py", line 541, in chmod
    self._request(CMD_SETSTAT, path, attr)
  File "/usr/local/lib/python3.5/dist-packages/paramiko/sftp_client.py", line 813, in _request
    return self._read_response(num)
  File "/usr/local/lib/python3.5/dist-packages/paramiko/sftp_client.py", line 845, in _read_response
    raise SSHException("Server connection dropped: {}".format(e))
paramiko.ssh_exception.SSHException: Server connection dropped: 
LourensVeen commented 5 years ago

Actually, we should probably also retry first, and only raise if we cannot reconnect...