Performance issue with throttle_requests

[JordiPolo]

I'm having weird results when using throttle_request.

With 16 users, 10 second run and throttle to 100 , I get a total of 780 requests. Lower than expected but ok. With 16 users, 10 second run and throttle to 1000, I get 720 requests With 16 users, 10 second run and throttle to 1000000, I get 640 requests

It is like keeping track of those bigger numbers take a toll on how much work can be done?

[jeremyandrews]

How many requests do you see if you disable the throttle altogether and run 16 users for 10 seconds?

[JordiPolo]

with it disabled: 780 so I guess I should have never expected 100 to affect

[JordiPolo]

When I do something like throttle 20 for 10 secs, I get a number near 200, so in that sense is working as expected.

[jeremyandrews]

I set up a local test between 2 vms on the same hardware, and ran with various configurations. I do not see any unexpected overhead when using the throttle. I recommend doing longer tests to obtain more consistent and reliable numbers.

For reference, I got the following numbers repeatedly running the test as follows (after running the load test for about ten minutes without any throttle to warm the caches of the Drupal site being load tested), adjusting the throttle value as necessary:

cargo run --release --example drupal_loadtest -- -H http://local.dev/ -v -t10s --throttle-requests 1000 | grep Aggregated | tail -3 | head -1

Testing locally, no throttle:

 Aggregated               |        28,907 |         0 (0%) |     2891 |    0.00
 Aggregated               |        30,923 |         0 (0%) |     3092 |    0.00
 Aggregated               |        29,216 |         0 (0%) |     2922 |    0.00
 Aggregated               |        29,730 |         0 (0%) |     2973 |    0.00

A throttle of 100:

 Aggregated               |           902 |         0 (0%) |    90.20 |    0.00
 Aggregated               |           904 |         0 (0%) |    90.40 |    0.00
 Aggregated               |           903 |         0 (0%) |    90.30 |    0.00
 Aggregated               |           902 |         0 (0%) |    90.20 |    0.00

A throttle of 1000:

 Aggregated               |         9,046 |         0 (0%) |   904.60 |    0.00
 Aggregated               |         9,049 |         0 (0%) |   904.90 |    0.00
 Aggregated               |         9,044 |         0 (0%) |   904.40 |    0.00
 Aggregated               |         9,046 |         0 (0%) |   904.60 |    0.00

A throttle of 10000:

 Aggregated               |        29,752 |         0 (0%) |     2975 |    0.00
 Aggregated               |        28,780 |         0 (0%) |     2878 |    0.00
 Aggregated               |        28,219 |         0 (0%) |     2822 |    0.00
 Aggregated               |        30,298 |         0 (0%) |     3030 |    0.00

A throttle of 100000:

 Aggregated               |        30,265 |         0 (0%) |     3026 |    0.00
 Aggregated               |        28,686 |         0 (0%) |     2869 |    0.00
 Aggregated               |        29,275 |         0 (0%) |     2928 |    0.00
 Aggregated               |        29,558 |         0 (0%) |     2956 |    0.00

Remember that the --throttle-requests value is an upper limit, not a lower limit. It will result in a number of requests equal to or less than the throttle, never more. In short tests (ie, 10 seconds) your averages will be heavily weighted downward by any overhead. I'd recommend running your tests for at least 10 minutes for more accurate numbers.

[jeremyandrews]

I'm unable to duplicate any performance issues. If you find a good way to duplicate this or are still concerned feel free to re-open with more details.

[JordiPolo]

I've run it again, 1 minute. In release mode, my previous test was in debug mode and I can not reproduce it either. I do think that compiling using release mode is the trick here. Sorry for the noise.

[jeremyandrews]

Thanks for following up. That makes sense, not enabling --release can drastically affect the performance profile.