add asyncio compatibility to the entire ape suite

[johnson2427]

Overview

Would love to move all suites to be integrated with asyncio. Ape requires a lot of waiting for responses, we could open up the main thread and speed up a lot of what we do for core functionality with the asyncio package. For web2 integrations and for database integrations, this would be a huge addition. Async capable packages are a huge plus.

Specification

synchronous calls:

def some_function(arg):
    return do_something(arg)

asynchronous calls:

async def do_something(arg):
   return await arg + 5

async def some_function(arg):
    return await do_something(arg)

(obviously really rudimentary case here) When combined with requests to APIs, async functions can free up the main thread, and allow for batching of calls, which will decrease the wait time for calls to complete by 10x in many cases. You would fetch for a response, while waiting for that API to receive, then get data, then send that data back, your thread would call the rest of the needed functionality until the await is met.

Dependencies

asyncio - https://docs.python.org/3/library/asyncio.html

For quick guides to asyncio:

• https://www.youtube.com/watch?v=2IW-ZEui4h4
• https://www.youtube.com/watch?v=GpqAQxH1Afc

Deeper Dive:

• https://bbc.github.io/cloudfit-public-docs/asyncio/asyncio-part-1.html

[antazoey]

Thought I had: Make it still synchronous out of the box but have async versions of the methods get automatically created.

@generate_async_version
def get_block()

will produce an extra method get_block_async which returns a co-routine.

This way, it is not a breaking change and noobs can still call the methods easily. Also synchronous and Python feels natural. But then the feature still exists and suites that need async can use those versions and even internally, we can async methods to speed up processing.

[johnson2427]

@unparalleled-js this is brilliant, and I love the concept. Completely agree on the comment "synchronous and Python feel natural." Python really took async hook, line and sinker following 3.10.5 (might be off on the version, but I think that's the one). And it does some cool stuff for API development. But getting used to development with async is definitely a steep challenge. If we can generate async functions like this, without having to completely tear down Ape, I'm in.

[fubuloubu]

Thought I had: Make it still synchronous out of the box but have async versions of the methods get automatically created.

@generate_async_version
def get_block()

will produce an extra method get_block_async which returns a co-routine.

This way, it is not a breaking change and noobs can still call the methods easily. Also synchronous and Python feels natural. But then the feature still exists and suites that need async can use those versions and even internally, we can async methods to speed up processing.

Can use this and have "ape as a library" work synchronous out of the box, and then "ape as an application" use the async API to integrate with click-async to speed up the overall application flow

[fubuloubu]

https://github.com/click-contrib/asyncclick

[fubuloubu]

See https://linear.app/apeworx/issue/APE-212/spike-add-asyncio-compatibility-to-the-entire-ape-suite for more info

[johnson2427]

do it!

[antazoey]

From @fubuloubu 👍🏻

one thing w.r.t. v1.0 to think about: the network context manager is not concurrency-friendly