search

locustio / locust

Write scalable load tests in plain Python 🚗💨
https://locust.cloud
MIT License
25.06k stars 3k forks source link

Wrong statistic of total request count #846

Closed liming-gd closed 6 years ago

liming-gd commented 6 years ago

Description of issue / feature request

Locust told me 249 requests had been made, but I got 327 records from service access log.

Expected behavior

I think locust should report that the completed requests is 327, but not 249.

Environment settings (for bug reports)

Steps to reproduce (for bug reports)

Just run the following command in shell:

locust -H http://10.44.147.237:8024 -f locustfile.py --no-web -c 50 -r 10 -n 200

lucustfile.py

# -*- coding: utf-8 -*-

from __future__ import absolute_import

import locust

class PerformanceTaskSet(locust.TaskSet):

    @locust.task(weight=1)
    def apply_resources(self):
        self.client.put(
            '/accounts/xxxx/projects/integration-test/resource_apply/',
            json={'cpu': 8, 'memory': 16}
        )

class PerformanceLocust(locust.HttpLocust):

    task_set = PerformanceTaskSet

    min_wait = 100
    max_wait = 1000
aldenpeterson-wf commented 6 years ago

Without setting not to reset stats, Locust will make requests while scaling up the users and then reset stats.

In 0.8.1 that is --no-reset-stats but will change in 0.9.0 https://github.com/locustio/locust/pull/674

gwshawsh commented 6 years ago

I have the same problem... It doesn't work with 0.9.0.

cgoldberg commented 6 years ago

@gwshawsh what doesn't work?

gwshawsh commented 6 years ago

@cgoldberg Locust shows 6283 requests , but I got 6329 records from access log. V0.9.0 without '--reset-stats' like this 'locust -f ./sale_test.py --no-web -c 800 -r 100 -t 60s' .

cgoldberg commented 6 years ago

are you actually seeing stats refresh after hatch, or do you just see a discrepancy in results and are guessing that is the cause?

could you be running an older version older than 0.9.0?

gwshawsh commented 6 years ago

THE LOGS

[root@tobacco-sit opt]# locust -f ./sale_test.py --no-web  -c 800 -r 100 -t 30s
[2018-09-27 10:32:19,727] tobacco-sit/INFO/locust.main: Run time limit set to 30 seconds
[2018-09-27 10:32:19,727] tobacco-sit/INFO/locust.main: Starting Locust 0.9.0
[2018-09-27 10:32:19,728] tobacco-sit/INFO/locust.runners: Hatching and swarming 800 clients at the rate 100 clients/s...
 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                              0     0(0.00%)                                       0.00

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                            181     0(0.00%)      50      34      97  |      45    0.00
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                            181     0(0.00%)                                       0.00

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                            366     0(0.00%)      51      34     108  |      46   55.50
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                            366     0(0.00%)                                      55.50

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                            562     0(0.00%)      59      34     199  |      50   74.50
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                            562     0(0.00%)                                      74.50

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                            741     0(0.00%)     126      34     865  |      58   80.67
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                            741     0(0.00%)                                      80.67

[2018-09-27 10:32:28,441] tobacco-sit/INFO/locust.runners: All locusts hatched: WebsiteUser: 800
 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                            963     0(0.00%)     331      34    1258  |      76   84.75
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                            963     0(0.00%)                                      84.75

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           1220     0(0.00%)     443      34    1258  |     150   88.10
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           1220     0(0.00%)                                      88.10

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           1446     0(0.00%)     396      33    1258  |     130  101.10
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           1446     0(0.00%)                                     101.10

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           1648     0(0.00%)     362      33    1258  |     120  106.10
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           1648     0(0.00%)                                     106.10

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           1880     0(0.00%)     335      33    1258  |     130  108.90
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           1880     0(0.00%)                                     108.90

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           2092     0(0.00%)     318      33    1258  |     140  111.70
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           2092     0(0.00%)                                     111.70

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           2320     0(0.00%)     303      33    1258  |     140  112.70
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           2320     0(0.00%)                                     112.70

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           2541     0(0.00%)     300      33    1258  |     150  111.30
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           2541     0(0.00%)                                     111.30

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           2770     0(0.00%)     295      33    1258  |     170  109.90
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           2770     0(0.00%)                                     109.90

 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           2851     0(0.00%)     300      33    1258  |     170  110.60
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           2851     0(0.00%)                                     110.60

[2018-09-27 10:32:49,619] tobacco-sit/INFO/locust.main: Time limit reached. Stopping Locust.
[2018-09-27 10:32:49,625] tobacco-sit/INFO/locust.main: Shutting down (exit code 0), bye.
[2018-09-27 10:32:49,625] tobacco-sit/INFO/locust.main: Cleaning up runner...
[2018-09-27 10:32:49,743] tobacco-sit/INFO/locust.main: Running teardowns...
 Name                                                          # reqs      # fails     Avg     Min     Max  |  Median   req/s
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                           2875     0(0.00%)     316      33    2950  |     170  103.50
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                           2875     0(0.00%)                                     103.50

Percentage of the requests completed within given times
 Name                                                           # reqs    50%    66%    75%    80%    90%    95%    98%    99%   100%
--------------------------------------------------------------------------------------------------------------------------------------------
 POST commitsalorder-1                                            2875    170    250    330    460    980   1100   1200   1200   3000
--------------------------------------------------------------------------------------------------------------------------------------------
 Total                                                            2875    170    250    330    460    980   1100   1200   1200   3000

Locust showed 2875 , but I got 3229 actually.

FlorEtcheverry commented 5 years ago

I'm running 0.11.0 version and I have the exact same problem.