While load testing and profiling aioboto3 we suspect that we found a bug where an async code is calling a synchronous function. This problem blocks the entire ioloop (i.e non boto calls are affected)
What I Did
Please see the sample code that demonstrates the issue.
Hi,
While load testing and profiling aioboto3 we suspect that we found a bug where an async code is calling a synchronous function. This problem blocks the entire ioloop (i.e non boto calls are affected)
Libraries used:
The main environment we use for testing (python3.6)
aioboto3 9.0.0
aiobotocore 1.3.1
boto3 1.17.49
botocore 1.20.49
Trivial example to demonstrate the issue:
"""Demonstrates the blocking issue with aioboto3."""
import asyncio
import datetime
import aioboto3
counter = 0
lock = asyncio.Lock()
async def retrieval_process():
global counter
session = aioboto3.Session()
async with session.resource('dynamodb', region_name='us-east-1',
endpoint_url='http://localhost:8000'
) as dynamo_resource:
pass
async with lock:
counter += 1
async def main(N):
t1 = datetime.datetime.now()
tasks = [retrieval_process() for _ in range(N)]
await asyncio.gather(*tasks)
while True:
async with lock:
if counter >= N:
break
await asyncio.sleep(0.1)
t2 = datetime.datetime.now()
print(f"N={N} total duration= {(t2 - t1).total_seconds()}")
if __name__ == '__main__':
loop = asyncio.get_event_loop()
loop.run_until_complete(main(2))
loop.run_until_complete(main(50))
Similar behaviour when debugging aiobotocore directly (python 3.8.10)
aiobotocore 2.1.0
botocore 1.23.24
The block is happening because aiobotocore calls synchronous functions of boto3.
As an example, think of the coroutine named _create_client (implemented in
aiobotocoro/session) which calls self._get_internal_component('endpoint_resolver')
def _get_internal_component(self, name):
# While this method may be called by botocore classes outside of the
# Session, this method should **never** be used by a class that lives
# outside of botocore.
return self._internal_components.get_component(name)
which eventually hits the get_component method in botocore.session which
executes the deffered (sync) function as can be seen here:
class ComponentLocator(object):
"""Service locator for session components."""
def __init__(self):
self._components = {}
self._deferred = {}
def get_component(self, name):
if name in self._deferred:
factory = self._deferred[name]
self._components[name] = factory()
# Only delete the component from the deferred dict after
# successfully creating the object from the factory as well as
# injecting the instantiated value into the _components dict.
del self._deferred[name]
try:
return self._components[name]
except KeyError:
raise ValueError("Unknown component: %s" % name)
The question
How we can assure the asynchronicity of both the Session and the create_client to eliminate the blocking.
This should be raised in the aiobotocore repo. That being said, it can be worked around for now by caching the client/resource by using something like an AsyncExitStack
Description
While load testing and profiling aioboto3 we suspect that we found a bug where an async code is calling a synchronous function. This problem blocks the entire ioloop (i.e non boto calls are affected)
What I Did
Hi,
While load testing and profiling aioboto3 we suspect that we found a bug where an async code is calling a synchronous function. This problem blocks the entire ioloop (i.e non boto calls are affected)
Libraries used:
The main environment we use for testing (python3.6)
Trivial example to demonstrate the issue:
Similar behaviour when debugging aiobotocore directly (python 3.8.10)
The block is happening because aiobotocore calls synchronous functions of boto3.
As an example, think of the coroutine named _create_client (implemented in aiobotocoro/session) which calls
self._get_internal_component('endpoint_resolver')
which eventually hits the get_component method in botocore.session which executes the deffered (sync) function as can be seen here:
The question
How we can assure the asynchronicity of both the
Session
and thecreate_client
to eliminate the blocking.