locustio / locust

Write scalable load tests in plain Python 🚗💨
https://locust.cloud
MIT License
25.06k stars 3k forks source link

Limit on Locust subclasses #1248

Closed neophyte57 closed 4 years ago

neophyte57 commented 4 years ago

Hi, I have reviewed issue 573 (https://github.com/locustio/locust/issues/573) and am following the recommendation: to have multiple HttpLocust subclasses and a corresponding TaskSet subclass for each HttpLocust subclass in my Locust file. But when I run Locust (version 0.13.5), this is the behavior I get

Arrangement of Locust file Behavior
combined_test21.py Both task sets executed
combined_test123.py Task sets #2 and #3 executed
combined_test231.py Task sets #3 and #1 executed
combined_test0231.py Task sets #0 and #2 executed
combined_test2310.py Task sets #2 and #3 executed

The number in the Python file name indicates the included test case #s (e.g. "combined_test0231.py" has the pair of TaskSet and HttpLocust classes for test case #0, followed by the pair for test case #2, etc.). I run the test within the Docker image locustio/locust:0.13.5 like this:

locust -f combined_test123.py --no-web -c 2 -r 1

So from my analysis, it seems at most two HttpLocust subclasses are executed. I was so surprised by this that I reviewed the Locust code (https://github.com/locustio/locust/blob/798156e5e202d89b64fab2f6a6c1cd33a1d7d0f2/locust/main.py#L406) and then my code. Yes, my classes satisfy the is_locust function (https://github.com/locustio/locust/blob/798156e5e202d89b64fab2f6a6c1cd33a1d7d0f2/locust/main.py#L337)

Note that these combined_test**.py files are actually the result of a "cat XXXXXXX.py YYYYYYYY.py > combined_test**.py" so the individual .py files are valid Locust files that are able to execute properly by themselves.

What am I doing wrong? Thanks.

heyman commented 4 years ago

So from my analysis, it seems at most two HttpLocust subclasses are executed.

Since you're using -c 2 locust will only be simulating two users (instances of Locust classes).

If you want simulate users that can execute many different TaskSets, you should have a single Locust subclass with a task_set that has multiple TaskSet subclasses as tasks.

Here's an example:

from locust import Locust, TaskSet, task, constant

class Tasks1(TaskSet):
    @task
    def some_task(self):
        print("task 1")

    @task
    def stop(self):
        self.interrupt()

class Tasks2(TaskSet):
    @task
    def some_task(self):
        print("task 2")

    @task
    def stop(self):
        self.interrupt()

class Tasks3(TaskSet):
    @task
    def some_task(self):
        print("task 3")

    @task
    def stop(self):
        self.interrupt()

class MyLocust(Locust):
    wait_time = constant(1)
    class task_set(TaskSet):
        tasks = {
            Tasks1: 1, 
            Tasks2: 1, 
            Tasks3: 1,
        }

For more info see: https://docs.locust.io/en/stable/writing-a-locustfile.html#tasksets-can-be-nested

neophyte57 commented 4 years ago

Thanks for the timely response. Yes, the reason for the observed behavior makes sense … doh!

Why I use the "cat" command is because I am converting these test cases from a different tool so I wanted to keep the same folder and file organization. I've been doing some experimenting based on your response and it seems I can cat the individual Locust files together, define a new HttpLocust class with a nested TaskSet that defines the distribution of TaskSets (directly or via configuration) and then when running Locust, specify this new HttpLocust class, like this:

` class OverallUnauthenticatedUser(HttpLocust):

class task_set(TaskSet):
    tasks = {
        TC1Behavior: Configuration.get_weight('TC1'), 
        TC2Behavior: 20, 
        TC3Behavior: 30,
        TC4Behavior: 40
    }

host = 'https://portal.foobar.ca/' 

`

FYI, Configuration is my own class. I didn't add the stop method in each TaskSet (e.g. TC1Behavior) and it seems to run the way I want to. By not defining, could this cause some problem?

heyman commented 4 years ago

I didn't add the stop method in each TaskSet (e.g. TC1Behavior) and it seems to run the way I want to. By not defining, could this cause some problem?

If your sub TaskSets never call the interrupt() method, it means that a once a simulated user starts executing one of those TaskSets, it'll keep running tasks from that TaskSet forever (until it's stopped) and never exit from it and pick a new sub TaskSet.