Open justdoit0823 opened 7 years ago
Wow, it's clear. Great job!!!
so asyncio is just better
@jonathan-daniel asyncio
is the native implementation. And the latest tornado
should have adopted the coroutine implementation and been compatible with asyncio
.
Coroutine
About the coroutine in Python, I have discussed at The evolution of coroutine in Python. In this article, I will focus on more details of the coroutine's differences between tornado and asyncio. Before diving into these details, let's start with two examples.
Use coroutine in tornado,
Use coroutine in asyncio,
Or new style with
async
andawait
,It's obvious that tornado uses
yield
expression rather thanyield from
expression, which makes the root difference. I will talk about the reason later.What's the real differences?
Because the
yield
syntax doesn't support controlling subiterator, the tornado must do much more work. Now take a look at how it works.How
gen.coroutine
works?Each function decorated with
gen.coroutine
returns aFuture
object, which is finished at the same time with the function. From the outermost function to the innermost function, there may be a lot of decorated coroutines are yielded. If the function is a common function, the future object is immediately finished with the function's return value as result. The coroutine will yield at the first unfinished future, which will be bound to the event loop. Once the future is finished, the coroutine continues the execution and yields at the next unfinished future point, and so on.During this process, the future's result is backward propagated from the function which returns the future to the function which yields the future. Future is the key point here. The execution pieces are linked together as a linear rope with a lot of futures. When the final future is finished, the whole process is done.
The
gen.coroutine
successfully simulates theyield from
syntax at application level with the generator object and future object. So we can write asynchronous functions as executing serially.How
asyncio.coroutine
works?If the function is written in
yield from
syntax, theasyncio.coroutine
just marks the function as an iterable coroutine at the bytecode level. The coroutine will yield as the same with thegen.coroutine
, but the value ofyield from
is directly returned to the outermost caller. For unfinished future, it's also bound to the event loop. Once the future is finished, the coroutine continues to the execution with the result passed from the caller and yield at the next unfinished future point, and so on.The future is returned from the inner subiterator to the outermost caller, and the result is passed from the outermost caller to the inner subiterator in the next execution. All these are driven by the
asyncio.Task
object.Summary
Above of all, the general processes of the two are the same, but the
asyncio.coroutine
doesn't force the decorated function to return future object, which also unifies the outermost coroutine as entry of the execution. With theyield from
syntax, the coroutine function's return value is automatically passed to the outer function, while tornado needs to do additional work to make the coroutine resumed at application level. Meanwhile there are some performance benefits of native syntax by reducing application level coroutine scheduling and unnecessary future objects.With the syntax
yield from
, it's easier for asyncio to implement cascaded coroutines which yield the future objects.Why tornado doesn't work with yield from?
I haven't asked this question in the community. But I have found somethings in tornado's change logs. There has been a long time since the minimal prototype of coroutine was introduced in tornado.
The history of tornado's coroutine
There is an example about writing an asynchronous handler in tornado.
At this moment, tornado uses the callbacks to combine relative asynchronous functions together based on the low-level event loop. With this style, the execution process of a function which contains asynchronous subroutines is divided into different parts with different contexts. It's flexible for users, but difficult for the readers.
gen
The
gen
module was added in version 2.1 on Sep 20, 2011. Whileyield from
was added in Python 3.3, which released on September 29, 2012. The new syntax is too late for tornado. At that time,yield
was the only mature choice. Withgen.engine
method decorating asynchronous handler, we can yield asynchronous functions wrapped withgen.Task
and imagine the task works serially. So we don't need to care about the callbacks, but the execution contexts are still separated. Although it's a good start for asynchronous functions with generator-based interface.gen.coroutine
The
gen.coroutine
decorator was added as an alternative togen.engine
in version 3.0 on Mar 29, 2013. It supports calling asynchronous function serially. And generator now yieldsFuture
objects rather thangen.Task
objects. The same example is written in a different way.This enhances about cascaded asynchronous functions. The
gen.Task
only solves one level yileding problem, and the later execution is still in the callback style way. At that time, theyield from
had been supported. But tornado still didn't use it. I think there may be two main reasons.The first is compatibility.
yield from
syntax is only supported in Python 3.3 and the later versions, whileyield
is supported both in Python 2 and Python 3. Adding any feature is needed to consider running both in Python 2 and 3, which is a huge cost and may make the development much harder.The second is separation. At the initial time, writing in different ways with different Python versions is not wise. For users, it's not convenient; for committers, it's hard to maintain. And all enhancements are only supported in Python 3, which benefits a little.
But the good news is that tornado 5.0 will support native coroutine. The details can be found at Tornado 5.0 Status. Running native coroutine in tornado,
Or
async
style,We can try the above codes with tornado on bdarnell/asyncio-future branch.
According to the history, tornado has been also improved a lot, and also keeps up to with the standard library implemention.
Reference
http://www.tornadoweb.org/en/stable/releases.html
https://docs.python.org/3/whatsnew/index.html
https://groups.google.com/forum/#!topic/python-tornado/7JNWKwCTvZs