Closed sebastian-wasilewski closed 6 years ago
The reason for random delay was to avoid all processes hitting the SSH server at once. If I put e.g. sleep(0.2) then all workers will sleep for 0.2s and then hit the SSH server.
I guess we could try to lower the rand(1.0) to 0.5s. I will do some testing with that. Also, another option would be to wrap that with an if statement, so the random delay would be only introduced if e.g. NP > 8.
Good point of course, I wasn't thinking clearly on the weekend ;). We can tune this going forward.
Parsyncfp is really a great tool! I have been trying to run it with high NP (64-128 and higher) and I noticed an issue that some of rsync processes often quit early at the start.
The issue seems to be limited to the initial rsync processes, so I think it may be caused by a large number of connections being opened with the remote SSH server at the same time. I have added a short, random delay before each process starts and so far I do not get those errors any more.
Please, would you accept adding this workaround to the master branch?