search

terricain / aioboto3

Wrapper to use boto3 resources with the aiobotocore async backend
Apache License 2.0
698 stars 73 forks source link

aioboto3 spawning more threads? #292

Closed VolkrB closed 1 year ago

VolkrB commented 1 year ago

Description

Implemented aioboto3 upload_fileobj in an async function.

What I Did

I might have a fundamental misconception about how async works, but as far as I understand there should only be one thread at a time (the main event loop) and all async functions including this:

async with sessions.client("s3") as s3:
   await s3.upload_fileobj(file, bucket, file2)

should not spawn any additional threads. However, every aioboto3 functions seems to spawn an additional thread for me. That being said, it still works concurrently. Is there a misconception that I have of how it works or is it maybe a bad implementation? If yes, i will share more code - I have replaced the above code with await asyncio.sleep(1) which does not spawn an additional thread

terricain commented 1 year ago

So at the end of it all there still exists botocore which is patched by aiobotocore and then used by aioboto3. I would hazard a guess theres probably some botocore stuff making some threads. That being said, I wouldn't be too concerned with them as the entire request path is all asyncified.

VolkrB commented 1 year ago

Cheers Terry and thanks for the quick response & awesome library. Closing