pallets / werkzeug

The comprehensive WSGI web application library.
https://werkzeug.palletsprojects.com
BSD 3-Clause "New" or "Revised" License
6.66k stars 1.73k forks source link

ProfilerMiddleware raises `ValueError: Another profiling tool is already active` #2909

Closed olejorgenb closed 3 months ago

olejorgenb commented 6 months ago

When using ProfilerMiddleware with threads (or - I assume this is a threaded-only issue) in python 3.12 ValueError: Another profiling tool is already active is spammed when the server handles concurrent requests

Server

from flask import Flask
from werkzeug.middleware.profiler import ProfilerMiddleware

app = Flask(__name__)

@app.route('/dummy')
def dummy():
    time.sleep(1.5)  # Emulate processing time
    return 'This is a dummy endpoint.'

import tempfile
tempdir = tempfile.gettempdir()
import time
app.wsgi_app = ProfilerMiddleware(app.wsgi_app, profile_dir=tempdir)

if __name__ == '__main__':
    app.run(debug=True, threaded=True)

Client

import threading

# Function to send request
def send_request():
    try:
        response = requests.get('http://127.0.0.1:5000/dummy')
        print(response.text)
    except requests.exceptions.RequestException as e:
        print(e)

# Create and start threads for concurrent requests
threads = [threading.Thread(target=send_request) for _ in range(10)]
for thread in threads:
    thread.start()
for thread in threads:
    thread.join()

Results in

127.0.0.1 - - [22/May/2024 15:24:53] "GET /dummy HTTP/1.1" 500 -
Traceback (most recent call last):
  File "temp2/lib/python3.12/site-packages/flask/app.py", line 1498, in __call__
    return self.wsgi_app(environ, start_response)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "temp2/lib/python3.12/site-packages/werkzeug/middleware/profiler.py", line 125, in __call__
    profile.runcall(runapp)
  File "temp2/lib/python3.12/cProfile.py", line 109, in runcall
    self.enable()
ValueError: Another profiling tool is already active

I assume https://github.com/python/cpython/issues/110770 is the root cause, but from https://github.com/python/cpython/issues/110770#issuecomment-1759986100 I gather profiling a threaded server never has worked properly.

Environment:

davidism commented 3 months ago

Seems like this isn't something we can solve.