locustio / locust

Write scalable load tests in plain Python 🚗💨
https://locust.cloud
MIT License
25.06k stars 3k forks source link

Make wait_time default to zero (vote up/down for this ticket please :) #1308

Closed cyberw closed 4 years ago

cyberw commented 4 years ago

Today all Locust classes need to have a wait_time specified. I suggest we change that to make no wait time the default. I think having no wait time is what a reasonable user would expect, and it significantly lowers the bar for new locust users.

Please vote up or down (or add any arguments you may have).

This would change the simplest possible locust file from:

from locust import Locust
from locust.wait_time import constant
class MyLocust(Locust):
     wait_time = constant(0)
     ...

To:

from locust import Locust
class MyLocust(Locust):
     ...
heyman commented 4 years ago

I'm currently voting down, and my reasoning is this that Locust's main philosophy has been that it's User-centered and not RPS-centered. For me, that is the power of Locust and what sets it apart from many other tools. That one declare real user-behaviour in code and chooses how many users to simulate, as opposed to choosing a number of RPS. With that mindset I don't think it makes sense to have a default of constant(0), since it shouldn't be the common case.

anuj-ssharma commented 4 years ago

Sorry, but I'll have to vote down for this, for a couple of reasons:

  1. A locust class represents a real user and a real user would not ever have a 0 second wait time. By setting the default to 0 seconds we are moving a locust away from representing a real user.
  2. If in a case the wait time is 0 seconds, then asking it to be written explicity would improve readability.

Just my 2 cents.

cyberw commented 4 years ago

Thanks for your input!

  1. In my mind a locust instance represents a single concurrent user. If I have a test case where I want to simulate ~100 concurrent users (maybe selected from a database/csv of 100.000 users) I would use 100 locusts, not 100.000. This very much represents "real users", but having a sleep after each iteration is not "more real" (because the time between one user leaving and another user arriving is not typically something that is a key feature of the load/realism)

  2. Having zero wait time is the standard in pretty much every other load test tool I have seen and I believe 99.9% of users would expect a zero sleep time between iterations if there isn't one specified so I think readability does not suffer at all.

I dont expect to convince anyone but these are my main arguments :)

jheld commented 4 years ago

Do we think there is a documentation issue?

Could we consider making a subclassed version of a locust/http client, in core/contrib, such that it comes loaded with the wait_time as described here?

I ran into a similar issue recently as I've recently reintroduced locust into our load test project this week. I did understand what a locust was (a user), but I had a mismatch in my mind about how often a given set of APIs would be hit by the user/locust.

Low wait time is still great to test! It's an excellent test, but agreed, if there is a polling API in your service/system for instance it likely isn't going to be hit every second or so (or less) by that specific user (locust), but instead more likely being hit every few (or even dozen) seconds, depending on many factors on a particular project of course.

heyman commented 4 years ago

Could we consider making a subclassed version of a locust/http client, in core/contrib, such that it comes loaded with the wait_time as described here?

That's one of my reasons for not wanting to make this change. It's sooo easy to just make a subclass in your own testscript, or even just a mixin class, if you want that behaviour:

class NoWaitTime:
    wait_time = constant(0)

class SuperDuperFastUser(NoWaitTime, User):
    @task
    def t(self):
        print("yay")

With the current implementation there's a risk that people forget to declare a wait_time in which case they get an error message that clearly states what's wrong. If we were to default it to zero, people who wouldn't declare a wait_time would not get an error and risk not actually realizing that one can set wait time. I think that would be really bad since the concept of declaring user behaviour is extremely important and the main use-case for Locust (and real users rarely have zero wait time).