Closed omikader closed 3 months ago
Werkzeug 2.3.0 is no longer supported. Try 3.0.x. You're also using Python 3.8, but 3.12 has significant speedups compared to that. And the dev server is not designed to be consistently performant. That said, there's not really anything we can do about this. If you want to use a profiler and figure out optimizations for a PR, that's fine, but it's too broad to just leave open otherwise.
Hi @davidism, thanks for the quick response! I had first noticed the issue when I upgraded to Python 3.12 and Flask/Werkzeug 3.0.3 so I spent a little time trying to isolate the change and landed on Werkzeug 2.2.3 -> 2.3.0. Here are the ab
and profiler results for the latest stable versions of Python + Flask, respectively.
% ab -n 100 http://127.0.0.1:8100/rolldice
This is ApacheBench, Version 2.3 <$Revision: 1903618 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/
Benchmarking 127.0.0.1 (be patient).....done
Server Software: Werkzeug/3.0.3
Server Hostname: 127.0.0.1
Server Port: 8100
Document Path: /rolldice
Document Length: 1 bytes
Concurrency Level: 1
Time taken for tests: 1.812 seconds
Complete requests: 100
Failed requests: 0
Total transferred: 17300 bytes
HTML transferred: 100 bytes
Requests per second: 55.20 [#/sec] (mean)
Time per request: 18.117 [ms] (mean)
Time per request: 18.117 [ms] (mean, across all concurrent requests)
Transfer rate: 9.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 0 0.1 0 1
Processing: 15 18 1.8 17 30
Waiting: 4 6 1.8 6 19
Total: 15 18 1.8 18 31
Percentage of the requests served within a certain time (ms)
50% 18
66% 18
75% 19
80% 19
90% 19
95% 20
98% 25
99% 31
100% 31 (longest request)
PATH: '/rolldice'
404 function calls (400 primitive calls) in 0.001 seconds
Ordered by: internal time, call count
List reduced from 194 to 30 due to restriction <30>
ncalls tottime percall cumtime percall filename:lineno(function)
1 0.000 0.000 0.000 0.000 {method 'disable' of '_lsprof.Profiler' objects}
1 0.000 0.000 0.000 0.000 {method 'write' of '_io.TextIOWrapper' objects}
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/routing/map.py:252(bind_to_environ)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/logging/__init__.py:298(__init__)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/sansio/request.py:159(args)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/wrappers/request.py:110(__init__)
4/1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/routing/matcher.py:79(_match)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/sansio/response.py:111(__init__)
1 0.000 0.000 0.001 0.001 /usr/local/lib/python3.12/site-packages/werkzeug/middleware/profiler.py:114(runapp)
2 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/app.py:401(create_url_adapter)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/app.py:1233(preprocess_request)
9/8 0.000 0.000 0.000 0.000 {method 'encode' of 'str' objects}
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/ctx.py:367(push)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/ctx.py:396(pop)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/routing/matcher.py:69(match)
1 0.000 0.000 0.000 0.000 /backend/./app.py:11(roll_dice)
1 0.000 0.000 0.001 0.001 /usr/local/lib/python3.12/site-packages/flask/app.py:1441(wsgi_app)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/app.py:1260(process_response)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/ctx.py:309(__init__)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/wsgi.py:233(__init__)
41 0.000 0.000 0.000 0.000 {built-in method builtins.isinstance}
5 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/local.py:310(__get__)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/app.py:1092(make_response)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/random.py:242(_randbelow_with_getrandbits)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/logging/__init__.py:1541(warning)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/ctx.py:251(push)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/sansio/request.py:118(__init__)
4 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/datastructures/headers.py:457(_str_header_value)
5 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/werkzeug/_internal.py:29(_wsgi_decoding_dance)
1 0.000 0.000 0.000 0.000 /usr/local/lib/python3.12/site-packages/flask/app.py:867(full_dispatch_request)
Given that the development server is not meant for production, I agree that it probably makes sense to close.
That doesn't match our benchmarks either, as there were significant speed increases to the code in Werkzeug 3, with the removal of bytes/str dual handling and the old urllib copy.
Thanks for following up @davidism! If it's not too much trouble, I'd love to take a look at your benchmarks so I can try to reproduce but no worries if not. Have a great week!
I am in the process of upgrading a Flask app and noticed a performance regression on the Werkzeug development server starting with version 2.3.0. To confirm, I created a simple app based on the OpenTelemetry guide and ran it as a Docker container.
I used Apache Benchmark to simulate requests on the same application running versions 2.2.3 and 2.3.0 of werkzeug, respectively. The former can handle 479.17 requests per second and the latter only handles 53.59.
I added the the
ProfilerMiddleware
but didn't notice anything out of the ordinary.On 2.2.3
On 2.3.0
I am aware that the development server provided by Flask is not intended for production use, so benchmarking it is perhaps a contrived exercise -- but I was surprised by these results and I couldn't find anything in the changelog or elsewhere online that explains the difference, hence the issue.
Thank you for the awesome project!
Environment: