network_timeout is not respected

[rokcarl]

I'm using network_timeout = 1.0 on a FastHttpLocust subclass, but the slow pages still succeed. I would expect that after 1.0s, the request would stop and another one would start. Additionally, I would expect the page load to count as an error. What I'm seeing though is that Locust is waiting for the page to load and counting it as a success.

I know there's a test in the codebase for this, but it only checks if there was an error in the slow page, but this could be due to other things.

Code for my Flask test web server `app.py`

import random import time from flask import Flask app = Flask(__name__) @app.route('/') def hello_world(): time.sleep(random.random() * 0.9) return 'Hello, World!' @app.route('/slow') def slow(): sleep = 3 + random.random() time.sleep(sleep) return f'Took me {sleep} s.' @app.route('/400') def bad_request(): return 'Bad Request for your data', 400

My `locust.py` file:

from locust import TaskSet from locust.contrib.fasthttp import FastHttpLocust from locust.wait_time import constant_pacing def index(l): l.client.get("/slow") class UserBehavior(TaskSet): tasks = {index: 1} class WebsiteUser(FastHttpLocust): task_set = UserBehavior wait_time = constant_pacing(1.0) network_timeout = 1.0 connection_timeout = 1.0

I run the web server with flask run --reload (need pip install flask) and run the locust without the GUI using locust -f locust.py --no-web -c 10 -r 10 -t 10s --host http://localhost:5000.

Here are the results

locust -f locust.py --no-web -c 10 -r 10 -t 10s --host http://localhost:5000 [2020-04-03 12:07:19,301] Roks-MacBook-Pro-2.local/INFO/locust.runners: Hatching and swarming 10 users at the rate 10 users/s (0 users already running)... [2020-04-03 12:07:20,229] Roks-MacBook-Pro-2.local/INFO/locust.runners: All locusts hatched: WebsiteUser: 10 (0 already running) [2020-04-03 12:07:20,306] Roks-MacBook-Pro-2.local/INFO/locust.main: Run time limit set to 10 seconds [2020-04-03 12:07:20,306] Roks-MacBook-Pro-2.local/INFO/locust.main: Starting Locust 0.14.5 Name # reqs # fails Avg Min Max | Median req/s failures/s -------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------- Aggregated 0 0(0.00%) 0 0 0 | 0 0.00 0.00 Name # reqs # fails Avg Min Max | Median req/s failures/s -------------------------------------------------------------------------------------------------------------------------------------------- -------------------------------------------------------------------------------------------------------------------------------------------- Aggregated 0 0(0.00%) 0 0 0 | 0 0.00 0.00 Name # reqs # fails Avg Min Max | Median req/s failures/s -------------------------------------------------------------------------------------------------------------------------------------------- GET /slow 10 0(0.00%) 3574 3100 4001 | 3400 0.00 0.00 -------------------------------------------------------------------------------------------------------------------------------------------- Aggregated 10 0(0.00%) 3574 3100 4001 | 3400 0.00 0.00 Name # reqs # fails Avg Min Max | Median req/s failures/s -------------------------------------------------------------------------------------------------------------------------------------------- GET /slow 10 0(0.00%) 3574 3100 4001 | 3400 0.00 0.00 -------------------------------------------------------------------------------------------------------------------------------------------- Aggregated 10 0(0.00%) 3574 3100 4001 | 3400 0.00 0.00 Name # reqs # fails Avg Min Max | Median req/s failures/s -------------------------------------------------------------------------------------------------------------------------------------------- GET /slow 20 0(0.00%) 3516 3095 4001 | 3400 1.43 0.00 -------------------------------------------------------------------------------------------------------------------------------------------- Aggregated 20 0(0.00%) 3516 3095 4001 | 3400 1.43 0.00 [2020-04-03 12:07:30,307] Roks-MacBook-Pro-2.local/INFO/locust.main: Time limit reached. Stopping Locust. [2020-04-03 12:07:30,309] Roks-MacBook-Pro-2.local/INFO/locust.main: Shutting down (exit code 0), bye. [2020-04-03 12:07:30,309] Roks-MacBook-Pro-2.local/INFO/locust.main: Cleaning up runner... [2020-04-03 12:07:30,309] Roks-MacBook-Pro-2.local/INFO/locust.main: Running teardowns... Name # reqs # fails Avg Min Max | Median req/s failures/s -------------------------------------------------------------------------------------------------------------------------------------------- GET /slow 21 0(0.00%) 3502 3095 4001 | 3400 1.91 0.00 -------------------------------------------------------------------------------------------------------------------------------------------- Aggregated 21 0(0.00%) 3502 3095 4001 | 3400 1.91 0.00 Percentage of the requests completed within given times Type Name # reqs 50% 66% 75% 80% 90% 95% 98% 99% 99.9% 99.99% 100% ------------------------------------------------------------------------------------------------------------------------------------------------------ GET /slow 21 3400 3600 3700 3800 4000 4000 4000 4000 4000 4000 4000 ------------------------------------------------------------------------------------------------------------------------------------------------------ None Aggregated 21 3400 3600 3700 3800 4000 4000 4000 4000 4000 4000 4000

You can see that all the requests were successful, but all of them should have failed and there should have been more requests, as each one should fail after 1.0 s.

Environment:

• OS: Mac OS
• Python version: Python 3.7.1
• Locust version: 0.14.5

[heyman]

I've added some additional checks to the test but that did not make it fail: 12c4d33b5a37cf71d22c7fc3d78b700f4778e974

[cyberw]

@rokcarl Can you try pulling latest master and see if it works? Since https://github.com/locustio/locust/pull/1304 , Locust classes must explicitly be declared abstract, so you may have ended up running the base FastHttpLocust class instead of WebsiteUser. I fixed FastHttpLocust in 461eafaf5355f2263d87bb4d1d36c5b808de2971

[heyman]

Can you try pulling latest master and see if it works? Since #1304 , Locust classes must explicitly be declared abstract, so you may have ended up running the base FastHttpLocust class instead of WebsiteUser.

Good catch!

[rokcarl]

I tried it from master, but now I get the following error:

[2020-04-06 17:46:46,736] docker-desktop/ERROR/locust.core: Cannot choose from an empty sequence
Traceback (most recent call last):
  File "/usr/local/lib/python3.7/site-packages/locust/core.py", line 260, in run
    self.schedule_task(self.get_next_task())
  File "/usr/local/lib/python3.7/site-packages/locust/core.py", line 419, in get_next_task
    return random.choice(self.locust.tasks)
  File "/usr/local/lib/python3.7/random.py", line 261, in choice
    raise IndexError('Cannot choose from an empty sequence') from None
IndexError: Cannot choose from an empty sequence

Which is weird because there's a test in locust to make sure that I get a different error if the tasks are empty. Have the same code as I had above. I also tried tasks = [], the same. I tried extending HttpLocust instead of FastHttpLocust, but it was the same again.

pip freeze

certifi==2020.4.5.1 chardet==3.0.4 click==7.1.1 ConfigArgParse==1.2 Flask==1.1.2 gevent==1.5a3 geventhttpclient-wheels==1.3.1.dev3 greenlet==0.4.15 idna==2.9 itsdangerous==1.1.0 Jinja2==2.11.1 locustio==0.14.5 MarkupSafe==1.1.1 msgpack==1.0.0 psutil==5.7.0 pyzmq==19.0.0 requests==2.23.0 six==1.14.0 urllib3==1.25.8 Werkzeug==1.0.1

I got this ☝️ by basically running pip install https://github.com/locustio/locust/archive/master.zip.

[cyberw]

Hmm. Not sure what is the problem (it could definitely be related to 1.0 changes), but if you use this syntax it should work:

from locust import task
class WebsiteUser(FastHttpLocust):
    @task 
    def index(self):
        self.client.get("/slow")
    wait_time = constant_pacing(1.0)
    network_timeout = 1.0
    connection_timeout = 1.0

[rokcarl]

Yes, this works and the timeout works as well, I ran it against my mock website with a timeout of 0.5 + random.random() (so between 0.5 and 1.5) and half of the requests were successful and half were a failure.

Thank you!