requests / requests-threads

🎭 Twisted Deferred Thread backend for Requests.
418 stars 27 forks source link

async/await example question #4

Open amejia1 opened 7 years ago

amejia1 commented 7 years ago

I'm not that familiar with this new AsyncSession object, but it looks to me like the async/await example doesn't actually perform those GET requests concurrently as you are awaiting for each one to return one by one. Should that example be this instead?

async def _main():
    for future in asyncio.as_completed([session.get('http://httpbin.org/get') for _ in range(100)]):
        print(await future)
dismantl commented 6 years ago

This is correct. Looking at a wireshark dump, I can confirm the async/await example sends the requests sequentially rather than concurrently.

When trying the suggested replacement function above however, I get the following error: builtins.AssertionError: yield from wasn't used with future

mjpieters commented 5 years ago

The session.request() method returns twisted Deferred objects; these can't really be treated as futures. You'd want to put all the deferreds into a DeferredList, I think. I'm not that versed in Twisted and their asyncio integration.

TheQWERTYCodr commented 1 year ago

you can just append the deferreds to a standard list and then await them all in a for loop. the for loop ends at roughly the same time as the slowest task, and is only slowed down by overhead from Python.