diux-dev / cluster

train on AWS
75 stars 15 forks source link

nexus: make file upload more user-friendly #58

Open yaroslavvb opened 6 years ago

yaroslavvb commented 6 years ago

Ideally:

Depends on https://github.com/diux-dev/cluster/issues/56 for prototyping solutions

yaroslavvb commented 6 years ago

Upload errors should be caught early, now get something like this

  File "launch.py", line 139, in <module>
    main()
  File "launch.py", line 132, in main
    launcher()
  File "launch.py", line 67, in launcher
    job.upload('jupyter_notebook_config.py', '~/.jupyter') # 2 step upload since don't know ~
  File "/Users/yaroslav/git/cluster/gpubox/../backend.py", line 132, in upload
    task.upload(*args, **kwargs)
  File "/Users/yaroslav/git/cluster/gpubox/../aws_backend.py", line 434, in upload
    sftp.put(local_fn, remote_fn)
  File "/Users/yaroslav/anaconda/envs/gpubox/lib/python3.6/site-packages/paramiko/sftp_client.py", line 727, in put
    return self.putfo(fl, remotepath, file_size, callback, confirm)
  File "/Users/yaroslav/anaconda/envs/gpubox/lib/python3.6/site-packages/paramiko/sftp_client.py", line 683, in putfo
    with self.file(remotepath, 'wb') as fr:
  File "/Users/yaroslav/anaconda/envs/gpubox/lib/python3.6/site-packages/paramiko/sftp_client.py", line 341, in open
    t, msg = self._request(CMD_OPEN, filename, imode, attrblock)
  File "/Users/yaroslav/anaconda/envs/gpubox/lib/python3.6/site-packages/paramiko/sftp_client.py", line 780, in _request
    return self._read_response(num)
  File "/Users/yaroslav/anaconda/envs/gpubox/lib/python3.6/site-packages/paramiko/sftp_client.py", line 832, in _read_response
    self._convert_status(msg)
  File "/Users/yaroslav/anaconda/envs/gpubox/lib/python3.6/site-packages/paramiko/sftp_client.py", line 861, in _convert_status
    raise IOError(errno.ENOENT, text)
FileNotFoundError: [Errno 2] No such file