amol-
commented
5 years ago Well, I think it will be curious to see. So feel free to go for it.
A few possible hints:
- The test should probably use minimal mode, as explained in https://turbogears.readthedocs.io/en/latest/turbogears/minimal/index.html#hello-world as it's the most similar thing to how the other frameworks are configured.
- The test should not use the
wsgirefWSGI server as in the TG examples, it should be onmeinheldto set a common ground with what's used in the other tests. - There are some routing tricks that can give a 10x benefit in this test case, but I'd say it's probably not the right thing to use them. Because we probably don't want to test how fast a TurboGears core developer himself can make the framework go, but how fast it is in default setup.
But there are tons of questions that arise :P It's a bit hard to interpret the results and might give a false sense of reality to readers. In practice it's 90% testing the WSGI Application Server used and 10% the webframework.
If you don't expose which WSGI server was used the results are pretty pointless. Like japronto uses asyncio while falcon uses meinheld so it isn't really a comparison of the two frameworks themselves.
I think it would make sense to split the web frameworks in families, like it makes little sense to test japronto with flask or flask with Django.
You probably want a distinction between async and threaded frameworks. Using japronto for a CPU bound application would be a pretty bad choice even though it obviously would perform better than flask on this test because in this test the webapp does nothing and thus is very far from being a CPU bound case in any way.
Also you probably want a distinction between full stack frameworks and microframeworks. Comparing Django to Flask is pretty unfair, there are tons of middlewares and features enabled in Django by default compared to Flask and they have a pretty big cost.
Depending on what's the goal of this benchmarks, they might wrongly lead someone to switch to the wrong technology without fully understanding what's really going on or they could get a nearly equivalent improvement by disabling features in what they are already using.
waghanza
Hi @amol- @pedersen,
Would you mind to add (or consent that we add) turbogears in
here?This project compare performance based on 3 routes :
GET/, SHOULD return a successful status code and an empty stringGET/user/:id, SHOULD return a successful status code and the idPOST/user, SHOULD return a successful status code and an empty stringRegards,