search

erdewit / nest_asyncio

Patch asyncio to allow nested event loops
BSD 2-Clause "Simplified" License
693 stars 79 forks source link

Working with tornado #23

Closed maartenbreddels closed 4 years ago

maartenbreddels commented 4 years ago

I ran into issues combining this package with tornado:

  File "/Users/maartenbreddels/miniconda3/envs/vaex37-test/lib/python3.7/site-packages/tornado/gen.py", line 712, in __init__
    if self.handle_yield(first_yielded):
  File "/Users/maartenbreddels/miniconda3/envs/vaex37-test/lib/python3.7/site-packages/tornado/gen.py", line 789, in handle_yield
    self.io_loop.add_future(self.future, inner)
  File "/Users/maartenbreddels/miniconda3/envs/vaex37-test/lib/python3.7/site-packages/tornado/ioloop.py", line 693, in add_future
    assert is_future(future)
AssertionError

The problem is tornado keeps a tuple of types that it considers futures. I have this workaround: https://github.com/vaexio/vaex/blob/293d510ef5c8b96ce75224e909a005599c96cc92/packages/vaex-core/vaex/asyncio.py#L5

But maybe you have a better idea for this, or maybe tornado is to blame for this? This line is the issue: https://github.com/tornadoweb/tornado/blob/712d61079defdad23b0a5e9fe0090b54e55cf7d0/tornado/concurrent.py#L49

erdewit commented 4 years ago

Asyncio ships with two different Futures: The default C-optimized one and a pure-Python one. Since the C future can't be patched, nest_asyncio needs the pure Python one and makes that one the default. So before nest_async.apply(), asyncio.Future will be the C-Future and after the Python one.

So instead of your patch, it should be possible to call nest_asyncio.apply() before importing tornado. Note that your patch also relies on the future having been patched before.

erdewit commented 4 years ago

A bit belated, but I've added this idea of patching tornado to nest_asyncio now as well. I did not fully realize before that there are a lot of use cases were tornado is already running and needs to be patched live.

Thank you very much for the code!

maartenbreddels commented 4 years ago

Great, happy to see this land in nest_asyncio!