parallel file copy stuck

ParallelSSH / parallel-ssh

Asynchronous parallel SSH client library.

https://parallel-ssh.org

GNU Lesser General Public License v2.1

1.2k stars 148 forks source link

parallel file copy stuck #302

Closed rvencu closed 3 years ago

rvencu commented 3 years ago

I try to copy files from remote with copy_remote_file as

cmds = dclient.copy_remote_file('/home/crawl/gpujob.zip', 'gpujob.zip')
joinall (cmds)

the problem is even the files arrive as expected, the script does not continue after the joinall line...

Not sure what I am doing wrong, there is no error message, just silent output

rvencu commented 3 years ago

actually, my bad, it appears stuck because I am downloading in parallel 28 files each 500MB

download speed is very slow... this seems to be another issue

rvencu commented 3 years ago

after some more tests, I noticed something weird though

I downloaded 30 x 170MB files. I also refreshed the folder with ls -lh command and noticed that files grew to their size in about 180 seconds. However, the joinall command only exited after 371 seconds with apparently nothing to do...

pkittenis commented 3 years ago

Thanks for the interest and report.

If you think there is an issue please provide:

Debug log output when issue occurs (pssh.utils.enable_debug_logger)
Code to reproduce

pkittenis commented 3 years ago

Closing as no steps to reproduce. Can be re-opened if the above is provided.