search

alexfernandez / loadtest

Runs a load test on the selected URL. Fast and easy to use. Can be integrated in your own workflow using the API.
MIT License
2.57k stars 209 forks source link

Concurrent Clients mismatch #226

Closed emil916 closed 10 months ago

emil916 commented 11 months ago

After updating loadtest, I am seeing a different value for CC in the end results. See below:

loadtest -t 5 -c 25 --rps 1000 "http://localhost:30000/fib?30"

Target URL: http://localhost:30000/fib?30 Max time (s): 5 Target rps: 1000 Concurrent clients: 23 Running on cores: 4 Agent: none

Completed requests: 4999 Total errors: 2 Total time: 5.182 s Mean latency: 3.7 ms Effective rps: 965

Percentage of requests served within a certain time 50% 3 ms 90% 5 ms 95% 7 ms 99% 12 ms 100% 30 ms (longest request)

-1: 2 errors

alexfernandez commented 10 months ago

Keep in mind that the latest version is creating concurrent clients on demand; if it cannot create 25 clients in the five assigned seconds it will report less, as in your case. Can you try with a longer time to see if it reaches 25? Thanks!

emil916 commented 10 months ago

I see. Honestly, I was actually thinking of how the cc (concurrent clients) option becomes useless when one specifies the rps. I feel like when the rps is specified loadtest should be the one to decide the most appropriate number of cc, based on the parameters (load, cpus, input size (file option) etc.).

I will test it out again and report the outcome here. If it matches my mental model I will close the issue, if that is okay with you. If not, feel free to close it.

Thank you!

Emil PhD student from GW

alexfernandez commented 10 months ago

Oh, you are right! I did not notice the --rps above. Actually it's explicitly mentioned in the documentation:

Note: the `--concurrency` option will be ignored if `--requestsPerSecond` is specified;
clients will be created on demand.
emil916 commented 10 months ago

Oh, then my bad. Is it a new thing or was it like this since old times? (I dont have the most update version yet).

I think this Note should be shown as a warning when one uses both options anyways.

alexfernandez commented 10 months ago

It's a new thing for versions 8+ I think. Instead of showing the warning I opted for displaying the actual number of concurrent clients, perhaps it could be made more evident :thinking:

emil916 commented 10 months ago

Alright, it all make sense now. Thanks! Closing the issue.