Closed z-image closed 12 years ago
ndenev
commented
12 years ago Initially the idea to limit this was the "--procs" option, which should limit the parallel SSH connections. I guess this would make the whole run slower, and in that case a sleep() might be beneficial. Have you measured run time with lower number of parallel processes?
z-image
commented
12 years ago --procs has its use, but in case you're executing something slower server side it adds up and slows the whole thing unnecessarily.
For example if you have 2000 servers, then mpssh -p 10 'sleep 10' will run in approximately 200 batches, thus finishing in 200 x (sleep 10 + ssh etc. overhead) == quite a bit more than 2000 seconds.
On the other hand mpssh -p 2000 'sleep 10' will finish in about 1 x (sleep 10 + ssh etc. overhead) seconds, which is ~15 seconds total time, but will hammer the workstation mpssh is being executed on as well as the dns cache. Adding --delay=10 (where 10 is milliseconds), would change finishing time from ~15 to ~35 seconds, while applying much less pressure through fork() and dns queries.
I understand this is somewhat corner case wishlist thing :-)
ndenev
commented
12 years ago Yes, this makes sense, the idea is good. I've commited a simple delay option in https://github.com/ndenev/mpssh/commit/da60eeb1bb0570aecfb2ab30878790d0282d5f8d
Great to see the recent activity on this project - polishing an already excellent tool!
One small thing I stumbled upon: if your server list gets in the thousands of lines, the initial burst of DNS requests and the 1000 fork()s are quite stressful. What works well for me is very small artificial delay before each ssh process start - e.g. usleep(10000), which is 10ms. For 1000 servers the initial burst of activity would be spread throughout ~10 seconds.
Maybe make it a command line option? E.g.: mpssh --delay=10
Of course, one can be tempted to think all kind of smarter algorithms - e.g. slow start where DNS response times / process creation times are measured and then using this information to chose suitable delay etc. I'm not convinced it is worth the effort though.