search

locustio / locust

Write scalable load tests in plain Python 🚗💨
https://locust.cloud
MIT License
25.06k stars 3k forks source link

Distributed master hangs #1058

Closed ghost closed 5 years ago

ghost commented 5 years ago

Description of issue

Running locust distributed with a master and two slaves (-no-web and -t 10s), master ends up in a hung state after both slaves terminate.

Expected behavior

When slaves terminate then the master should shutdown and terminate.

Actual behavior

Master never terminates and must be killed with Ctrl-C

Environment settings

Steps to reproduce (for bug reports)

Have checked for a hung thread but not detected. Master and slaves are running in separate Docker containers initiated by Compose. How can debug flow in locust so I can see what master is doing that it won't terminate.

A simple configuration which does nothing (no web I/O) works properly. But when issuing web and docker requests master hangs

cgoldberg commented 5 years ago

are you specifying -t on the master?

ghost commented 5 years ago

Yes I have tried -t on the master. All slaves are told to terminate and they go through tears-owns, but then they are stuck. It seems the problem goes beyond distributed locust. I have setup a docker-compose that runs three independent containers with the same locust load test. The time is set to 5 minutes and once the time expires only one container terminates and the other two are stuck.
At this point I am going to avoid docker-compose and run each container in the background.

ghost commented 5 years ago

Problem appears to be caused by Python Docker SDK