How to use async await

[bdevel]

I'm unable to find any documentation or examples online for how to use Python's await feature. Since it's syntax feature it's not clear how to call it. I'm working with the Telethon Python package which returns <coroutine object TelegramBaseClient.connect at 0x137fc4e60> coroutine objects.

Browsing the asyncio and Coroutines docs it wasn't obvious how to start a loop and get the results out synchronously.

Any help would be appreciated.

[jjtolton]

This is the first time it's been requested, so behavior is currently undocumented. I'll take a look at it when I can.

On Wed, Jul 1, 2020 at 7:55 PM Tyler notifications@github.com wrote:

I'm unable to find any documentation or examples online for how to use Python's await feature. Since it's syntax feature it's not clear how to call it. I'm working with the Telethon Python package which returns <coroutine object TelegramBaseClient.connect at 0x137fc4e60> coroutine objects.

Browsing the asyncio https://docs.python.org/3/library/asyncio-eventloop.html and Coroutines https://docs.python.org/3/library/asyncio-task.html docs it wasn't obvious how to start a loop and get the results out synchronously.

Any help would be appreciated.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/clj-python/libpython-clj/issues/106, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPJX46EG4SFRO3QPDX2E63RZPEFJANCNFSM4OOHS7OA .

[jjtolton]

Unfortunately unless you can persuade @Chris Nuernberger chris@techascent.com to add forms for async def and await, the best that can be done in the short term is to write creative wrappers. Some inspiration can be found here: https://stackoverflow.com/questions/54553907/is-there-a-way-to-call-an-async-python-method-from-c

Some time ago we had an internal discussion and no one could name a major library that relied on asyncio. This is the first time I've encountered this. If I can find a generic Python technique to accomplish async def and await programmatically then we can write a wrapper for it, but nothing is immediately coming to mind.

On Wed, Jul 1, 2020 at 8:21 PM James Tolton jjtolton@gmail.com wrote:

This is the first time it's been requested, so behavior is currently undocumented. I'll take a look at it when I can.

On Wed, Jul 1, 2020 at 7:55 PM Tyler notifications@github.com wrote:

I'm unable to find any documentation or examples online for how to use Python's await feature. Since it's syntax feature it's not clear how to call it. I'm working with the Telethon Python package which returns <coroutine object TelegramBaseClient.connect at 0x137fc4e60> coroutine objects.

Browsing the asyncio https://docs.python.org/3/library/asyncio-eventloop.html and Coroutines https://docs.python.org/3/library/asyncio-task.html docs it wasn't obvious how to start a loop and get the results out synchronously.

Any help would be appreciated.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/clj-python/libpython-clj/issues/106, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACPJX46EG4SFRO3QPDX2E63RZPEFJANCNFSM4OOHS7OA .

[cnuernber]

James hit on it. Technically you can do it already, you do have to transliterate the C example in the stackoverflow question in to python-clj code but that is doable. It is a matter of finding the attributes like __await__ and transforming the code. Note this is similar to what the Clojure async go macro does now.

That coroutine object probably has an await attribute. What happens if you list all the attributes on the coroutine object using py/att-type-map.

[jjtolton]

So defining an async function will be easy enough. I'd be happy to add this into our sugar namespace, but I'm not sure how to accomplish the await functionality.

# thank you Peter Norvig
# https://www.udacity.com/wiki/cs212/unit-5-code
def decorator(d):
    "Make function d a decorator: d wraps a function fn."
    def _d(fn):
        return update_wrapper(d(fn), fn)
    update_wrapper(_d, d)
    return _d

@decorator
def defasync(f):
    async def _f(*args, **kwargs):
        return f(*args, **kwargs)
    return _f

@defasync
def foo(a, b):
    return a + b

x = foo(1, 2)

>>> x
# <coroutine object foo at 0x7f30b1cf3e60>

The question of how to do await seems much trickier -- unless that class definition above did what we wanted.

[jjtolton]

Reading the article, is seems like it would be very possible to define an awaitable programmatically using the the techniques above. I have no idea if the async context will translate as intended though in something hypothetical like:

(py/defasync foo [a b]
    (+ a b))

(py/defasync bar [n]
    (dotimes [_ n] 
        (py/let-await [x (foo n n)]
            (println x))))

but worth a shot I suppose.

My guess is that this would have to be a very complex macro akin to the clojurescript implementation of go blocks and <!.

[bdevel]

The result of (py/att-type-map (py. client :connect)) is:

;; <coroutine object TelegramBaseClient.connect at 0x136a91a70>
{"__await__" :method-wrapper,
 "__class__" :type,
 "__del__" :method-wrapper,
 "__delattr__" :method-wrapper,
 "__dir__" :builtin-function-or-method,
 "__doc__" :none-type,
 "__eq__" :method-wrapper,
 "__format__" :builtin-function-or-method,
 "__ge__" :method-wrapper,
 "__getattribute__" :method-wrapper,
 "__gt__" :method-wrapper,
 "__hash__" :method-wrapper,
 "__init__" :method-wrapper,
 "__init_subclass__" :builtin-function-or-method,
 "__le__" :method-wrapper,
 "__lt__" :method-wrapper,
 "__name__" :str,
 "__ne__" :method-wrapper,
 "__new__" :builtin-function-or-method,
 "__qualname__" :str,
 "__reduce__" :builtin-function-or-method,
 "__reduce_ex__" :builtin-function-or-method,
 "__repr__" :method-wrapper,
 "__setattr__" :method-wrapper,
 "__sizeof__" :builtin-function-or-method,
 "__str__" :method-wrapper,
 "__subclasshook__" :builtin-function-or-method,
 "close" :builtin-function-or-method,
 "cr_await" :none-type,
 "cr_code" :code,
 "cr_frame" :frame,
 "cr_origin" :none-type,
 "cr_running" :bool,
 "send" :builtin-function-or-method,
 "throw" :builtin-function-or-method}

I did try running the __await__ function like so (py. (py. client :connect) __await__) but you just get back another co-routine <coroutine_wrapper object at 0x137f36bd0>.

FWIW, I've experimented with creating an event loop (def loop (asyncio/new_event_loop)) and (asyncio/gather (py. client :connect) :loop loop) which returns a Future, and (py. loop :run_until_complete (py. client :connect)) which returns nil but I have no experience with asyncio so not sure if I'm even close.

Looking like the best thing to at this point is a wrapper and/or py/run-simple-string.

[bdevel]

Looks like the Telethon library can actually be turned syncronous with a magic import statement import telethon.sync - very subtle. Found it in the Telethon asyncio docs. Thanks for the help.

[alpox]

I tried around a bit and found that I can make things work in a relatively simple way - although I'm not sure about the implications and best practices here as I'm fairly new with Clojure in general.

By copying over coroutines as CompletableFuture and CompletableFuture back to python as coroutines (futures) its possible that the python event loop understands the Clojure side and vice-versa. This allows an integrated/mixed use through a helper like Promesa (In the gist imported as pp):

https://gist.github.com/alpox/8777e2bd9b305845832021af9c95deaf

In case anyone is interested ;-)

[jjtolton]

@alpox whoa that's awesome!! More than an interesting way to tackle asyncio in libpython-clj, it's also a great example of how to extend libpython-clj in user space (and for PRs)! We should add this to the documentation for sure!