No (real) performance improvements when running --parallel

[davidlinse]

Hi..

I tried to make use of the --parallel option and observed some unexpected results.. Running this two simple steps with version 1.0.6 against github and localhost

   ...
  "steps": [
    {
      "type": "get",
      "url": "http://sebuilder.github.com/se-builder/"
      // or
      // "url": "http://localhost/"
    },
    {
      "type": "pause",
      "waitTime": "5000"
    }
  ]
  ...

with the following setup:

./node_modules/.bin/se-interpreter \
--parallel={n}  #1,2,3,4
--driver-host=127.0.0.1 \
--driver-port=4444 \
--browser-browserName=chrome \
./local.json ./local.json ./local.json ./local.json

gives something like:

        github.com |  localhost

--parallel=1
real    1m2.193s   |  0m16.710s
user    0m0.373s   |  0m0.372s
sys     0m0.067s   |  0m0.065s

--parallel=2
real    0m34.493s  |  0m13.394s
user    0m0.370s   |  0m0.369s
sys     0m0.066s   |  0m0.066s

--parallel=3
real    0m48.121s  |  0m24.202s
user    0m0.370s   |  0m0.381s
sys     0m0.065s   |  0m0.062s

--parallel=4
real    0m36.870s  |  0m19.623s 
user    0m0.370s   |  0m0.375s
sys     0m0.068s   |  0m0.068s

I'd expect decreasing execution times with increased instances ? Can somebody confirm this ?

Note: The local selenium server is configured to accept up to 5 sessions.

best regards ~david

[Zarkonnen]

Hmm, what happens when you use long scripts? It's possible that overhead is currently dominating execution time, though I admit this result is rather suboptimal!

[davidlinse]

I'll try to give it a try an report back.

[davidlinse]

Running full_example.json four times gives me something like:

--parallel=1
real 74.930s 
real 74.688s 
average: 74.809 ..

--parallel=2 
real 44.975s 
real 43.625s 
average: 44.3 ..

--parallel=3 
real 41.863s 
real 43.932s 
average: 42.8975 ..

--parallel=4 
real 28.789s 
real 28.536s 
average: 28.6625 ..

Hmm.. Pretty identical values compared to above.. Maybe the bottleneck is my machine.. (?) Will have to test on a better machine next week.

~david