locustio / locust

Write scalable load tests in plain Python 🚗💨
MIT License
24.64k stars 2.96k forks source link

Locust slaves slowing dying and then coming back to life? #190

Closed ghost closed 9 years ago

ghost commented 10 years ago

So I was running a load test with 10 locust slaves and I was monitoring their CPU activity with top. At the beginning of the load test, all slaves were active. Then they went "down" one by one (i.e., their CPU activity went to 0%) until only 1 was alive and active. And after 30 seconds, they all came back to life (i.e., positive CPU activity).

I ran this several times and it happened each time.

My setup is very simple:

The slaves query the API through a HTTP POST request. They input a key and get a value in return.

Other remarks:

I don't understand what was going on. Is it possible that the network caused this? What else could cause such a behaviour? (a bug in the API?)

heyman commented 10 years ago

Hi!

Hmm, hard to tell what might be wrong without more info.

Do you see any errors in the slave's output/log? How many users were you simulating? Have you increased the maximum number of open files/sockets (what does ulimit -a say?)? Do you have some firewall that might have temporary stopped the requests?

heyman commented 10 years ago

Any update on this?

ghost commented 9 years ago

I am closing this ticket as I am not able to reproduce the problem. Thank you for your help.