Comparison of performance test results between locust and jmeter

[vwin]

Description of issue / feature request

I use jmeter and locust do the same perf test ,but difference of the test results between locust and jmeter is huge. Jmeter result: Throughput 50/sec Locust result: Requests 100/s

How to define "requests" in locust? what's the difference between throughput and requests?

• OS:
• Python version:3.6
• Locust version:0.8.1

Steps to reproduce (for bug reports)

TBD - example code appreciated

[gunesmes]

From Jmeter's page it explains it as number of requests per second, which means the same with Locust.

Throughput is calculated as requests/unit of time. The time is calculated from the start of the first sample to the end of the last sample. This includes any intervals between samples, as it is supposed to represent the load on the server. The formula is: Throughput = (number of requests) / (total time).

[vwin]

@gunesmes I do a perf test ,but i am confused about the result e.g: I send 10000 requests, locust result is avg:122ms, min: 110ms, max:333ms, req/s:16.2 test start time is 19:41:02, end time is 19:51:21 , total time is 619s , req/s = 10000/619 = 16.2 but why total time is 619? 619 is less than 110ms*10000 = 1110s,
How does locust calculate this time 619s? I think the Minimum time should not less than 1110s

[aldenpeterson-wf]

Locust's number of requests is defined by the number of locusts, the time the ops take, and the taskset weighttimes.

If you have 10 locusts with a min/max weight of 1000ms between tasks and each operation takes 1 second, you will make 5 requests/second - each locust will:

• Make a request (t=0 through t=1000)
• Wait 1000ms
• Make another request (t=2000 through 3000)

The paradigm for Locust is simulating user workflows. If you want to check "maximum req/sec" you will want to reduce the minimum/maximum weight times to close to 0ms.

[cgoldberg]

what's the difference between throughput and requests

throughput is a rate.. like requests/sec.

[cgoldberg]

the reason you see a massive difference is that there is a configurable wait time between requests in locust and you likely dont have something similar in your jmeter tests. if you set the wait time to 0 you would see more similar results.

closing this issue since there is nothing actionable here.

[vwin]

@aldenpeterson-wf @cgoldberg thank u very much~

[vwin]

@gunesmes @aldenpeterson-wf @cgoldberg I do a test, set min_wait=0 & max_wait=0, then I sent 10000 requests, locust version:0.8.1 , use this cmd: locust --no-web -c1 -n10000 -r1 --csv=../test.csv -f perftest.py --host=myhost --no-reset-stats i print some log in perftest.py , finally , i find locust totally sent 5000 requests