Waitress Flask Server Cannot Handle Requests gives timeout

[Kalanamith]

We have a simple Flask app which has a single end point

Alive view

class Alive(MethodView):
    """
    Alive end point implementation
    """

    def get(self):

        response = Response(status=200, mimetype="application/json; charset=utf-8")
        response.charset = "utf-8"
        return response

Serving flask app

serve(app, host=host, port=port)

If a client do requests to this end point it takes 12 to more than 20 seconds to response and event gets timeout errors and can frequently see this warning

WARNING:waitress.queue:Task queue depth is 28
WARNING:waitress.queue:Task queue depth is 29
WARNING:waitress.queue:Task queue depth is 30
WARNING:waitress.queue:Task queue depth is 31
WARNING:waitress.queue:Task queue depth is 32
...

This end point will get called several times within a second

Why cannot it serve the request ?

[stevepiercy]

Did you search the issue tracker before creating a new issue?

[Kalanamith]

Yes @stevepiercy

[stevepiercy]

What makes your issue different from one of those issues? We need that information to help you.

[mmerickel]

I'm gonna need a fully reproducible example app, not just a flask view snippet. I'm unable to re-create any issues with a simple WSGI app and I don't know flask very well to build out the rest of your example. What settings/adjustments are you using in waitress, if any? What version of waitress? Are you really not returning a body at all? How many requests are coming in at once?

[Kalanamith]

Waitress settings

serve(app, host=0.0.0.0, port=5000, threads=8)

The code which I can share

class Alive(MethodView):
    """
    Alive end point implementation
    """

    def get(self):

        response = Response(status=200, mimetype="application/json; charset=utf-8")
        response.charset = "utf-8"
        return response

Try to do more than 50 api calls for this end point within a second

[mmerickel]

Thank you but I need something I can run. This is stuff that was in your original post.

[mmerickel]

I ran several experiments and don't see anything weird about this. If you have 50 requests per second on 8 threads, then your requests need to take less than some number of milliseconds (8 threads / 50 rps / 1000 ms/s = 160ms in a perfect world with no GIL and perfect parallelism which you will not have, so it'll be worse) or you will be falling behind and this is when the warning is displayed. It depends on the execution time of your stack (which is not 0 even in your simple scenario).

If I run a test with 50 rps (separate connections, no pipelining) on 8 threads using the code below, I see no queue depth warnings at all. If I add a time.sleep(0.1) inside the view then I see warnings between 1-8 holding steady but not constantly increasing. This is what I would expect.

import waitress

def app(environ, start_response):
    start_response('200 OK', (
        ('Content-Type', 'application/json; charset=utf-8'),
    ))
    return []

if __name__ == '__main__':
    waitress.serve(app, threads=8)

Please try to give more information about your platform / environment (waitress version I asked above, as well as OS, python version, etc) if you are seeing things differently than I describe - especially with my example.

[digitalresistor]

With no further feedback, I am going to close this issue.