JoeDog / siege

Siege is an http load tester and benchmarking utility
GNU General Public License v3.0
5.91k stars 386 forks source link

If I want to repeat the test several times but need to wait in each round of test, how should I run it ? #169

Open qiulang opened 4 years ago

qiulang commented 4 years ago

Say I wan to run the test twice, but need to wait for 1 second between them, I had thought I should run it as

siege -r 2 -c 10 -d 1 https://xxxx

I set my nginx as following so I though all test should be pass, But to my surprise the Failed transactions number from 1 to less than 10.

limit_req_zone $request_uri zone=ruri:1m rate=10r/s
limit_req zone=ruri burst=10 nodelay;

But if I use a simple script to sleep 1 second between the test, both test pass as I expect.

#! /bin/bash
siege  -r 1  -c 10 http://xxx
sleep 1
echo "wait for 1 sec to test again"
siege  -r 1  -c 10 http://xxx

So what does -d mean exactly and how to put a delay between each round of test ? The reason I want a delay is to test the rate I set in nginx

JoeDog commented 4 years ago

-d is a delay between each users request. If you have -c 10 -d 0.5 then each of the ten users will delay 0.5 seconds after each request.

On Wed, Feb 19, 2020 at 3:08 AM qiulang notifications@github.com wrote:

Say I wan to run the test 2 twice, but wait for 1 second between them, I had thought I should run it as

siege -r 2 -c 10 -d 1 https://xxxx

I set my nginx as following so I though all test should be pass, But to my surprise the Failed transactions numbers from 1 to less than 10.

limit_req_zone $request_uri zone=ruri:1m rate=10r/s limit_req zone=ruri burst=10 nodelay;

But if I use a simple script to sleep 1 second between the test, both test pass as I expect.

So what does -d mean exactly and how to put a delay between each round of test ? The reason I want a wait is to test rate set in nginx

! /bin/bash

siege -r 1 -c 10 http://xxx sleep 1 echo "wait for 1 sec to test again" siege -r 1 -c 10 http://xxx

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/169?email_source=notifications&email_token=ABJRHZUS4IGYPHI6IFWTZVDRDTSGZA5CNFSM4KXUBES2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4IORU6VQ, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZWWILXUXTTIZ5XZQ5LRDTSGZANCNFSM4KXUBESQ .

qiulang commented 4 years ago

Hi thanks for replying my question. But I still don't understand what you meant :$ "If each of the ten users will delay 1 seconds after each request." when siege -r 2 -c 10 -d 1 https://xxxx why did the Failed transactions number run from 1 to less than 10 in my several times.

I had thought they should all past b/c I set rate=10r/s in nginx. After the first 10 users test run, wait 1 second, then nginx should be able to process them again.

What am I missing here ?

BTW when I run siege -r 2 -c 10 -d 1 I feel like I always get more than 10 requests simultaneously print their result and I feel a pause then I get the remaining ones, like the following

HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 503 0.00 secs: 197 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 503 0.00 secs: 197 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 503 0.00 secs: 197 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.00 secs: 23 bytes ==> GET /elog

JoeDog commented 4 years ago

I don't know what rate=10r/s means. I assume that's a nginx directive

If you need nginx to handle 10 concurrent users, then I would slowly step up its capacity until it passes. It's a good idea to have more handlers then you need.

On Wed, Feb 19, 2020 at 10:15 AM qiulang notifications@github.com wrote:

Hi thanks for replying my question. But I still don't understand what you meant :$ "If each of the ten users will delay 1 seconds after each request." when siege -r 2 -c 10 -d 1 https://xxxx why did the Failed transactions number run from 1 to less than 10 in my several times.

I had thought they should all past b/c I set rate=10r/s in nginx. After the first 10 users test run, wait 1 second, then nginx should be able to process them again.

What am I missing here ?

BTW when I run siege -r 2 -c 10 -d 1 I feel like I always get more than 10 requests simultaneously print their result and I feel a pause then I get the remaining ones, like the following

HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 503 0.00 secs: 197 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 503 0.00 secs: 197 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 503 0.00 secs: 197 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.01 secs: 23 bytes ==> GET /elog HTTP/1.1 200 0.00 secs: 23 bytes ==> GET /elog

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/JoeDog/siege/issues/169?email_source=notifications&email_token=ABJRHZWSG2HHE2JWIUWGTUDRDVEJHA5CNFSM4KXUBES2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEMIFXBA#issuecomment-588274564, or unsubscribe https://github.com/notifications/unsubscribe-auth/ABJRHZX3CP62L7NZLWG56KDRDVEJHANCNFSM4KXUBESQ .

qiulang commented 4 years ago

Hi it is nginx directive, please check here https://www.nginx.com/blog/rate-limiting-nginx/

Rate – Sets the maximum request rate. In the example, the rate cannot exceed 10 requests per second.

That is why I used -d 1 hoping to wait for 1 second then nginx can process the request again. Unfortunately the result I got seems to show -d 1 was not what I need.

So if I need a pause between each round of test what should I do ?

BTW, nginx used burst to handles concurrent users (in my opinion)