Concurrent requests are taking more time then single request for python api code(using flask + waitres WSGI server)

[SagarPalyal]

When I am hitting api code with a single request, the response is coming back in 1 second approximately. Thats fine. As I am using waitress wsgi server , my flask code is handling concurrent request but problem is coming on performance. Whenever more than one concurrent requests are hitting api code, response time is getting increased to 1.5 sec to 2 sec to so on depends upon how many concurrent requests are hitting the api. More the concurrent requests more the response time for all requests. Provided threaded concept is working fine and currently waitress has been configured with 50 threads.

[stevepiercy]

Without a reproducible example, there is no way to help you, other than suggest that you profile the full request and response cycle to determine exactly where is the bottleneck. If you were using Pyramid, I would suggest turning on the pyramid_debugtoolbar, but I believe Flask has something similar.

Waitress itself can handle thousands of requests per second, so any performance issues are almost always within your app's code or slow database queries.

[SagarPalyal]

Hi stevepiercy, my code something looks like below. Please have a look. If I am doing something wrong here please let me know. Thanks for suggesting flask profiler as well , I will look into it also.

`from flask import Flask, request, g, jsonify from waitress import serve import json import time

app = Flask(name)

@app.before_request def before_request(): g.start = time.time()

@app.route('/', methods=['POST', 'GET']) def home(): data = json.loads(request.data.decode('utf-8'))

some statements

result = es.search() #fetching some data from elasticsearch
#some statements
data['New_key'] = 'some value'  #updating data
data['New_key1'] = 'some value' #updating data
return jsonify(data)

@app.after_request def after_request(response): diff = time.time() - g.start return response

@app.errorhandler(Exception) def handle_exception(e): return response

if name == 'main': serve(app,host='0.0.0.0' port=5000)```

[SagarPalyal]

Also flask profiler debugtoolbaar is of no use because this works when render HTML responses but in my case, response is in json format. :(

[digitalresistor]

This sort of question really belongs on the Pylons mailing list, and not in the bug tracker. Waitress's threads don't run concurrently unless a thread gives up the global interpreter lock, this means that even as you increase the threads linearly you won't see an increase in workload processed linearly. Any work that is being completed for a single thread will block all the others from doing useful work. There will be an absolute limit at which point you won't see any benefit what so ever from the increase thread count.

In fact I am willing to bet that even though your thread count is 50, your CPU for the Python process is mostly idle (and will be at most using a single CPU core unless it's spending time in C code that is not behind the global interpreter lock). I don't know what your es.search() function is doing, but I am guessing that it's not entering a function that allows a thread to give up the global interpreter lock and allowing other threads to utilize that time to do processing (and or is expensive anyway).

Waitress's threading concept is not ideal, but mostly works if you are heavily waiting for IO, as during that time a thread will not block the global interpreter lock, this works really well for apps that are reading/writing from disk/reading from postgresql, not so well for applications that do heavy processing in Python itself.

Speaking of, jsonify may also be a culprit here, depending on the size of the data you are turning into JSON a lot of time may be spent converting to JSON, you may try using a different JSON library in its stead.

I will ask that you take this to the Pylons mailing list though: https://pylonsproject.org/community-support.html