Closed greed2411 closed 5 years ago
Thanks for the idea.
A PR
will be very :heart:
Okay, just saw the results, it's at the bottom. And what I gave was just a synchronous version of tornado because there is no db querying or API calls, if that's there as a test, Im sure it would be comparable to Sanic
. Could you guys try to include that in a test as well?? Ik the test is just for routing, but still wanted to ask.
@greed2411 the result will be display on the next release
@tbrand If you don't have time ;-) I can help you releasing this project :stuck_out_tongue_winking_eye:
@greed2411 tornado
can run on gunicorn
is there any diff between gunicorn and built-in server ?
DISCLAIMER : not a python guy
For example, in ruby
world the recommended set-up is to use puma
or unicorn
(or else) but not the same server as in development mode
@benoitc As the gunicorn
author, you'll probably explain this like anyone :heart:
@waghanza yes it can. If I'm not wrong the in built server is wsgi.
I guess someone could rewrite it for gunicorn. Should I look over it?
@greed2411 sure, as flask
also does ( werkzeug
) but not recommended for production
I can make the PR
(making uniform python implementations), just want to ask before :stuck_out_tongue_winking_eye:
@waghanza if you are familiar with tornado then please go ahead and do the PR. Please do mention me in the PR. I've never tried gunicorn
I wanna have a look. I'd be happy to help anyway possible. Also thanks.
@greed2411 gunicorn
is made to be a production-ready app server in python
;-)
@waghanza Cool. Can't wait to see the results. I just wanna see tornado
on top of django
nothing else.
After cloudifying results, I'm sure it will be :stuck_out_tongue_closed_eyes:
I am working on a digitalocean version of the benchmarks
Also one more request @waghanza could you try it with tornado 5?
@greed2411 sure. I intend to fix any version (in any languages)
Can't wait to see the results. :P
@waghanza last chance for tornado, could you try these, run the results and lemme know?
http://www.tornadoweb.org/en/stable/wsgi.html
I still can't settle with the fact that tornado is slower than flask and django for this task.
Also thanks.
@waghanza the tornado script is performing actually synchronously, there's nothing async about it because we don't have and external api calls or db calls.
but, if I may, can you ask the core contributor, the one man army, bdarnell in the gitter or any personal means to look over the tornado script. Since you are a maintainer, he could take the time and give suggestions.
I don't have the face to ask him, because of me, the script is sitting at 44th place (last position T_T )
He could look at the table from the vibora PR, and let you know whether wsgi
should be used, if yes whether we should use tornado.wsgi
inside the script, or whether those routes can be handled asynchronously.
could you ask him, please?
@greed2411 do not be afraid ... you can do it :stuck_out_tongue:
@bdarnell could you check our tornado
implementation https://github.com/tbrand/which_is_the_fastest/blob/master/python/tornado/server.py. The purpose is to rank frameworks and I'm afraid this implementation is not at tornado
's advantage
Looks fine to me. You don't want to use tornado.wsgi if you don't have to. The one thing you may want to change (depending on how this server is run and if the other frameworks are run in comparable configurations) is to replace http_server.listen(port)
with http_server.bind(port); http_server.start(0)
to run in multi-process mode.
@bdarnell thanks for advice. As we target production-ready app, do you recommand this one, https://github.com/waghanza/which_is_the_fastest/blob/python_standardization/python/tornado/server.py, or the actual is better ?
PS : The goal of this implementation is to run with gunicorn
in front
The only goal is to have 3 endpoints :
GET
on /
POST
on /user
GET
on /user/(\d+)
I use Tornado without gunicorn in production; I don't know what gunicorn adds here.
Log uniformization ;-) same start-up script fot tornado
/ flask
...
We need to implement what is the best practices for tornado
folks :stuck_out_tongue_winking_eye:
@benoitc Do you recommand using gunicorn
on top of tornado
?
The reason we are considering because I still can't believe the fact that flask and Django are out performing tornado's requests/sec, because of gunicorn. Previously before gunicorn, they didn't even make the list. (Tornado's at the bottom, 44th)
Btw I'm the one who wrote the original script.:relieved:
@greed2411 I'm sure that tornado
will climb after having production-ready results :stuck_out_tongue_closed_eyes:
Here is a list of what we SHOULD use (as we are target
performance :stuck_out_tongue_winking_eye:)
gunicorn
@see https://docs.djangoproject.com/fr/2.0/howto/deployment/wsgi/waitress
@see http://flask.pocoo.org/docs/1.0/tutorial/deploy/#deploy-to-productionbuilt-in
@waghanza wowowow. Amazing. I never stopped believing in torando
, thank the heavens its better than django
and flask
, btw where can I see the actual results? I wanna see the margin by which we defeated django
, also I would like to see the modified tornado
script.
@greed2411 so impatient ... :stuck_out_tongue_winking_eye:
@greed2411
Framework (Middleware) | Requests / s | Latency | 99 percentile | Throughput |
---|---|---|---|---|
sanic | 4081.00 | 273718.00 | 2009469.33 | 2.40 MB |
japronto | 47752.67 | 21337.33 | 32104.00 | 19.44 MB |
flask | 1143.33 | 86290.00 | 104374.33 | 0.88 MB |
django | 1176.33 | 243895.00 | 4071113.33 | 0.78 MB |
tornado | 7054.00 | 137939.67 | 294950.67 | 5.21 MB |
Yussssssh. I feel overpowered now. I'll go and rub this onto every friend of mine at University. Thanks. I'm looking forward to see these results on readme soon. Good luck and thanks once again @waghanza. You made my dream come true.
But wait. Tornado is faster than Sanic????
You sure???? So the only thing on top of tornado is Japronto and Vibora huh?
@greed2411 I was NOT using https://github.com/Pylons/waitress at full speed
Framework (Middleware) | Requests / s | Latency | 99 percentile | Throughput |
---|---|---|---|---|
sanic | 4289.00 | 291132.67 | 2697882.00 | 2.57 MB |
japronto | 53348.00 | 20165.33 | 38664.00 | 21.56 MB |
flask | 1149.67 | 85609.33 | 104615.00 | 0.85 MB |
django | 1008.00 | 307748.00 | 4666972.33 | 0.70 MB |
tornado | 7219.33 | 133642.33 | 274002.33 | 5.49 MB |
:warning: tornado
5
seems to be faster (results are still on local docker)
Lel. Django slowed down. ๐๐๐.
Nonetheless tornado 5
is awesome.
@bdarnell I ran some tests to see if gunicorn
add some performances, and I've found
Appserver | Requests / s |
---|---|
Tornado | 5640.67 |
Gunicorn | 1835.33 |
Gunicorn (reuse port) | 1719.00 |
Tornado (reuse port) | 7051.00 |
PS : This is mainly for information (mine) and implementation in this project :stuck_out_tongue_winking_eye:
PSS : Result are fluctuating due do docker
but the ratio
is an indicator
what is that test? each worker in gunicorn should run 1 tornado server so i would be quite surprised it would be slower...
On Monday, July 2, 2018, Marwan Rabbรขa notifications@github.com wrote:
@bdarnell https://github.com/bdarnell I ran some tests to see if gunicorn add some performances, and I've found Appserver Requests / s Tornado 5640.67 Gunicorn 1835.33 Gunicorn (reuse port) 1719.00 Tornado (reuse port) 7051.00
โ You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/tbrand/which_is_the_fastest/issues/131#issuecomment-401918954, or mute the thread https://github.com/notifications/unsubscribe-auth/AAA4osAYqL9qbLGcZGnc2WCeTTqCKGGuks5uCn35gaJpZM4SRyqb .
-- Sent from my Mobile
Hi @benoitc, I ran some benchmarks and I have
Framework (Middleware) | Requests / s | Latency | 99 percentile | Throughput |
---|---|---|---|---|
tornado (code) | 7030.67 | 136106.00 | 326479.33 | 5.18 MB |
tornado-reuseport (code) | 7933.00 | 123524.33 | 316101.33 | 5.37 MB |
gunicorn (code) | 1181.00 | 858316.00 | 5904862.00 | 0.93 MB |
gunicorn-reuseport (code) | 1135.00 | 774897.67 | 2993184.33 | 0.88 MB |
Gunicorn run with : gunicorn --bind 0.0.0.0:3000 --workers `nproc` -k tornado server:app
the question is how you run the benchmark. There should be no reason to have a request slower since a tornado server will run on each worker process with gunicorn. Can you share the code of it?
sure, it is this repo, we use wrk
to request server
:warning: Still on (local) docker
, so result are fluctuating (cloudify this in planed)
These new results don't look right to me. I haven't done any performance comparisons myself but I expect Sanic to be faster than Tornado. I also expect that gunicorn -k tornado
performs similarly to plain Tornado. Maybe other python configurations also need to be updated to use the right number of processes?
@bdarnell effectively, I do not understand the diffs between gunicorn
and plain tornado
may you try running the code above https://github.com/tbrand/which_is_the_fastest/issues/131#issuecomment-402066677 ?
I don't understand that difference either. But unfortunately good benchmarking is a lot of hard work, and barely have time to benchmark my own software, much less somebody else's.
@bdarnell sure, that's could also be a argument for this project ^^
trying to compare tools is a burden, but it we can gather community, I am sure it will be helpful :stuck_out_tongue_winking_eye:
Absolutely. It's useful work, I'm just not volunteering to do it :)
no problem at all, anybody will always be welcome, at least when we have time
Try out the non blocking library for python. Tornado