locustio / locust

Write scalable load tests in plain Python 🚗💨
https://locust.cloud
MIT License
25.1k stars 3k forks source link

WebInterface stop updating RPS if Users are doing nothing #2970

Open andreabisello opened 2 weeks ago

andreabisello commented 2 weeks ago

Prerequisites

Description

let's suppose we are spawning 5 HttpUser and every HttpUser makes a self.client.get calls. The webinterface update itself. but let's suppose the task has a counter, and after the counter is passed, no self.client.get calls are done. le't suppose every user will has a counter set to 3, and if >3, do nothing. increasing the counter +1 at every execution, we expect 5*4=20 calls.

as you see, 20 calls are done, the test still running image but taking a look at the ux, it looks like it's still doing 2.33 rps, that's not real, because no one is doing nothing the ux should show 0 image

Command line

locust -f .\github_webux.py

Locustfile contents

from locust import HttpUser
import time

class WebuxUser(HttpUser):

    host = ""

    def on_start(self) -> None:

        self.counter = 0

    @task
    def to_something(self):
        if self.counter > 3:
            return
        self.client.get("https://docs.locust.io/")
        self.counter += 1
        time.sleep(1)

Python version

3.11

Locust version

2.32.0

Operating system

windows 11

andrewbaldwin44 commented 2 weeks ago

Hey @andreabisello, thanks for reporting! I think this is a known issue where Locust is using the total aggregated RPS report["total_rps"] = total_stats["current_rps"] instead of the current RPS. This is something we wanted to fix at some point right @cyberw?

cyberw commented 2 weeks ago

Wasn't that issue just for avg response times? Anyways, definitely something for us to have a look at, but maybe not right now. PRs welcome tho :)

andreabisello commented 2 weeks ago

I would like to try to help, i'm gonna download the project and take a look =)

cyberw commented 2 weeks ago

Have a look at https://docs.locust.io/en/stable/developing-locust.html if you haven't already!

andreabisellonamirial commented 6 days ago

@cyberw Hi, it's me, with the company account :) sorry, i have a problem.

I have followed your link, everything is installed, up and running. I have

poetry run locust

running my locust file,

PS C:\src\locust> poetry run locust
[2024-11-15 22:13:27,621] CTO-BISELLO/INFO/locust.main: Starting Locust 0.0.1.dev5438
[2024-11-15 22:13:27,621] CTO-BISELLO/INFO/locust.main: Starting web interface at http://localhost:8089 (accepting connections from all network interfaces)

and i have

yarn dev

opening a locust web ui .... on http://localhost:4000/dev.html


  VITE v5.4.8  ready in 326 ms

  ➜  Local:   http://localhost:4000/
  ➜  Network: use --host to expose
  ➜  press h + enter to show help

but how i'm suppose to obtain a ux handling the load test done by http://localhost:8089 to see what i'm doing modifying the ux?

sorry if the question sounds stupid, i had developed vuejs some years ago, that project it's a bit complicated for me, but i would like to help.

thankyou.

andrewbaldwin44 commented 6 days ago

Hi @andreabisello, I think for this ticket you won't have to modify the frontend code, the issue is most likely here: https://github.com/locustio/locust/blob/master/locust/web.py#L463, where you can see the current_rps is always set to the total. You could change this to any number and you would see the change reflected in the frontend.

If you do want to make some changes to the frontend, it's probably more useful to run yarn watch. This will rebuild the frontend code anytime it detects changes (simply refresh the page).

If you have an other questions just let us know don't be shy!

andreabisellonamirial commented 5 days ago

@andrewbaldwin44 I think the problem is that environment.runner.stats https://github.com/locustio/locust/blob/c0c7fd0ed6e8aa7ed65dec7806a8d125a78b8661/locust/web.py#L448 are updated only when runner catch a on_request event https://github.com/locustio/locust/blob/c0c7fd0ed6e8aa7ed65dec7806a8d125a78b8661/locust/runners.py#L111 and that is fired (i imagine) only when self.client execute a call.

for that reason the ux stops to update itself : because self.client stops to make calls.

if i'm right, maybe we should introduce a timer that update stats even if calls are not be done?

andrewbaldwin44 commented 5 days ago

I believe the issue is that the webui is displaying the "current_rps", which is an average:

def current_rps(self) -> float:
    ...
    reqs: list[int | float] = [
        self.num_reqs_per_sec.get(t, 0) for t in range(slice_start_time, int(self.stats.last_request_timestamp) - 2)
    ]
    return avg(reqs)

It takes into account all of the requests since the beginning of the test, so the RPS will never go back to zero.

The part that creates confusion is that the StatsEntry class is used for both request statistics and aggregated statistics, however I don't believe that the current_rps method was ever intended to be used for the aggregated statistics.

If we rewind time a little bit (commit 7cbc85ed), the RPS that was displayed in the webui was done like so:

# stats.py
@property
def reqs_per_sec(self):
    timestamp = int(time.time())
    reqs = [self.num_reqs_per_sec.get(t, 0) for t in range(timestamp - 10, timestamp)]
    return avg(reqs)

# web.py
@app.route('/stats/requests')
def request_stats():
    from core import locust_runner
    stats = []

    total_requests = 0
    total_rps = 0

    for s in locust_runner.request_stats.itervalues():
        total_requests += s.num_reqs
        total_rps += s.reqs_per_sec

        stats.append([
            s.name,
            s.num_reqs,
            s.avg_response_time,
            s.min_response_time,
            s.max_response_time,
            s.reqs_per_sec,
        ])

    stats.append(["Total", total_requests, "", "", "", round(total_rps, 2)])

    return json.dumps(stats)

So calculating the total average RPS for the last 10 seconds. I think the solution here then would be to simply re-introduce this logic in some way