Open markgdawson opened 5 years ago
Here is some code, using self.sftp_client, which would recursively create directories given either a remote file or remote directory path. Happy to submit a pull request if you want.
def create_directory(self, remote_path, is_directory=False):
"""
recursively create directories if they don't exist
remote_path - remote path to create.
is_directory - specifies if remote path is a directory
"""
dirs_ = []
if is_directory:
dir_ = remote_path
else:
dir_, basename = os.path.split(remote_path)
while len(dir_) > 1:
dirs_.append(dir_)
dir_, _ = os.path.split(dir_)
if len(dir_) == 1 and not dir_.startswith("/"):
dirs_.append(dir_) # For a remote_path path like y/x.txt
while len(dirs_):
dir_ = dirs_.pop()
try:
self.sftp_client.stat(dir_)
except:
self.log(f'creating directory {dir_}')
self.sftp_client.mkdir(dir_)
At the start of the push_file method of the Slurm class, the sftp_client.mkdir paramiko method is called. This method will only create a single directory level which does not exit.
Recursively creating directories would allow the channel to be used to easily push files to nested directories.