Native coroutine compatibility moving forward

[PatTheMav]

As I plan to adopt native coroutines (async def/await) in my Tornado app soonish, I'd like to know if there are any plans to update/adapt TorMySQL (or even Torpeewee, I use both) to use those? It might even require a fork as that would make both only Python 3.5+ compatible, but it's definitely the faster option moving forward (even more so when Tornado 5 uses the asyncio loop).

There are some gotchas mentioned in the official Tornado docs regarding that switch and I'm not aware how much of it concerns your projects.

Would appreciate hearing your opinion on this.

[snower]

You can directly use, tornado will automatically convert native coroutines (async def / await) and tornado future, tormysql and Torpeewee are support pymysql 3.5 +.

[PatTheMav]

Forgive me if I'm mistaken, I'm a bit confused as the docs state something else.. 😕

Also it was stated on the Tornado mailing list that starting with Python 3.5+ async/await should be used instead of the coroutine generator (as it's about 28x faster).

But that requires replacing all instances of

@tornado.gen.coroutine
def FUNC(...)
    yield SOME_OTHER_COROUTINE()

    return someValue # instead of raise gen.Return starting with Python 3.3+

with

async def FUNC(...)
    await SOME_OTHER_COROUTINE()

    return someValue

I was under the impression that requires changing it inside the TorMySQL and Torpeewee source code as well, otherwise I'd have to use e.g. await tornado.gen.convert_yielded(UserModel.get(UserModel.field == value)) which introduces the generator overhead again (IIRC Tornado coroutines return a Future while async def returns a coroutine object that needs be awaited).

[snower]

async def run():
    transaction = await db.transaction()
    try:
        datas = [d for d in (await Test.use(transaction).select().limit(10))]
        print(datas)
    except:
        await transaction.rollback()
    else:
        await transaction.commit()

You can use this directly, tornado inside the auto conversion.

[PatTheMav]

Thank you for your quick replies @snower - much appreciated.

I did some digging and tested some TorMySQL and Torpeewee boilerplate code and both work fine with current Tornado versions and native coroutines.

Further things I've found (and written down so I can make sense of it):

• concurrent.futures.Future does not implement __await__ so they are not compatible with await - tornado.concurrent.Future does however (and that's what TorMySQL uses a lot), so that's part of why it's working out-of-the-box (also Tornado flags the generator functions as native coroutines in newer Python versions to make them compatible with async/await
• Using an asyncio loop requires wrapping all await calls with tornado.platform.asyncio.to_asyncio_future (of course), as that's replacing concurrent.futures.Future for newer Python versions (and which does implement __await__).

So as for Python 3.5 no changes to the project(s) are needed and for Tornado 5 it might require "wrapping" every legacy Future into an asyncio-style future (but the decorator might take care of that).

Still leaves the 28x speed increase on the table though (but Tornado 5 might remedy that).

(Feel free to close the issue then).