search

scroogie / pdsh

Automatically exported from code.google.com/p/pdsh
0 stars 0 forks source link

pdsh slow when using a few thousand hosts and genders #59

Closed GoogleCodeExporter closed 8 years ago

GoogleCodeExporter commented 8 years ago 
pdsh is slow to generate the list of hosts when using a few thousand hosts 
selected via genders. I first noticed this on zin, but it is easy to recreate 
with a simple genders file like "foo[1-3000] bar". The results for pdsh-2.30 
are:

max = 1000  :  foo[1-1000]  real 0m0.790s user 0m0.778s sys 0m0.010s
max = 2000  :  foo[1-2000]  real 0m3.136s user 0m3.110s sys 0m0.019s
max = 3000  :  foo[1-3000]  real 0m7.043s user 0m6.980s sys 0m0.045s
max = 4000  :  foo[1-4000]  real 0m12.558s user 0m12.472s sys 0m0.057s
max = 5000  :  foo[1-5000]  real 0m19.728s user 0m19.579s sys 0m0.104s
max = 6000  :  foo[1-6000]  real 0m28.414s user 0m28.238s sys 0m0.112s
max = 7000  :  foo[1-7000]  real 0m38.689s user 0m38.411s sys 0m0.190s
max = 8000  :  foo[1-8000]  real 0m50.599s user 0m50.265s sys 0m0.220s
max = 9000  :  foo[1-9000]  real 1m3.908s user 1m3.293s sys 0m0.470s
max = 10000 : foo[1-10000]  real 1m19.859s user 1m19.083s sys 0m0.594s

While the results for pdsh-2.20 are:

max = 1000  :  foo[1-1000]  real 0m0.009s user 0m0.007s sys 0m0.001s
max = 2000  :  foo[1-2000]  real 0m0.015s user 0m0.011s sys 0m0.005s
max = 3000  :  foo[1-3000]  real 0m0.022s user 0m0.017s sys 0m0.005s
max = 4000  :  foo[1-4000]  real 0m0.035s user 0m0.027s sys 0m0.004s
max = 5000  :  foo[1-5000]  real 0m0.038s user 0m0.032s sys 0m0.007s
max = 6000  :  foo[1-6000]  real 0m0.051s user 0m0.039s sys 0m0.010s
max = 7000  :  foo[1-7000]  real 0m0.055s user 0m0.048s sys 0m0.008s
max = 8000  :  foo[1-8000]  real 0m0.067s user 0m0.049s sys 0m0.016s
max = 9000  :  foo[1-9000]  real 0m0.079s user 0m0.061s sys 0m0.017s
max = 10000 : foo[1-10000]  real 0m0.090s user 0m0.076s sys 0m0.013s

Note how for v2.30, 3000 hosts takes 7.043s while for v2.20 it takes 0.022s.

Original issue reported on code.google.com by mark.gro...@gmail.com on 31 Oct 2013 at 8:17

GoogleCodeExporter commented 8 years ago 
This issue was updated by revision 3c17946e6211.

Add a test to the testsuite to ensure that a genders -g query on
the order of 10000 hosts returns in reasonable time (<1s)

Original comment by mark.gro...@gmail.com on 31 Oct 2013 at 11:16

GoogleCodeExporter commented 8 years ago 
This issue was closed by revision e149fe78edea.

Original comment by mark.gro...@gmail.com on 31 Oct 2013 at 11:16