Support await expressions at the top level, like in asyncio REPL

[pavel-kirienko]

One often needs to invoke async callables in doctests:

>>> async def foo(): ...
>>> await foo()

Currently, this is not allowed because an await anywhere but inside an async function is a syntax error.

One commonly suggested solution is to use asyncio.run:

>>> async def foo(): ...
>>> asyncio.run(foo())

This approach is rarely useful, however, because asyncio.run() will terminate the event loop and finalize all coroutines at exit, which makes it impossible to persist state across invocations (which makes this solution mostly useless except for the most trivial use cases).

Since recently there is an asyncio REPL available that allows top-level await; it can be invoked simply as python -m asyncio. It is desirable to support this syntax in doctests, too.

[Erotemic]

I think adding this functionality is an important next step for xdoctest.

This is a similar issue I got in another repo: https://github.com/Erotemic/mkinit/issues/14 Unfortunately, the fix is a bit more complex here. In mkinit, all I had to do was support parsing async definitions, (which can be added here without too much trouble). But running an await inside a doctest seems a bit more challenging. Adding to this, I don't have much experience with async (I tend to use concurrent.futures.ThreadPoolExecutor instead).

I don't think I will have the time to take on this issue in the near future, so any help would be greatly appreciated. The place in the code where code in the doctests are compiled and executed is here: https://github.com/Erotemic/xdoctest/blob/main/xdoctest/doctest_example.py#L639

I believe this is where the major changes would need to take place.

If anyone does want to work on this, send me a ping, and I'll give you my slack/discord/email, so you can ask me any questions in real time. (or just make a PR and post updates, and I'll respond as best I can)