Can't run async client inside tests

[yovelcohen]

• Package Name: azure-data-tables
• Package Version: 12.5.0
• Operating System: "macOS-14.5-x86_64-i386-64bit"
• Python Version: 3.11.7

I'm trying to test my Fastapi app that connects to the azure storage table, I've set up a local azurite for the tests and I run my tests using unittest's IsolatedAsyncioTestCase.

The connection to azurite is setup during the fastapi's lifespan function. When the code reaches the actual test after that, every operation I'm trying to do, ends up with a "RuntimeError: Event loop is closed".

To Reproduce Steps to reproduce the behavior:

1. I've tried creating the Table Client in multiple ways:

   
   from azure.data.tables.aio import TableServiceClient, TableClient

conn_str = cls.settings.AZURE_STORAGE_CONNECTION_STRING connection_verify = False if 'https://127.0.0.1:10002/devstoreaccount1' in conn_str else True loop = asyncio.get_event_loop() connector = TCPConnector(loop=loop) session = ClientSession(connector=connector, trust_env=True) cls._client = TableServiceClient.from_connection_string( conn_str=conn_str, connection_verify=connection_verify, session=session ) await cls._client.get_service_properties() # for ping

2. the  `await cls._client.get_service_properties()` works there. it's only later, outside the lifespan and in the actual test case, that every operation fails (with a recursive error)

await srt_table_client.get_entity(1)

File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/data/tables/_generated/aio/operations/_operations.py", line 526, in query_entity_with_partition_and_row_key pipeline_response: PipelineResponse = await self._client._pipeline.run( # pylint: disable=protected-access ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 221, in run return await first_node.send(pipeline_request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/policies/_redirect_async.py", line 73, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/data/tables/aio/_policies_async.py", line 141, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 72, in send raise e File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 69, in send response = await self.next.send(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/_base_async.py", line 106, in send await self._sender.send(request.http_request, request.context.options), ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/data/tables/aio/_base_client_async.py", line 316, in send return await self._transport.send(request, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/azure/core/pipeline/transport/_aiohttp.py", line 280, in send result = await self.session.request( # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/aiohttp/client.py", line 581, in _request conn = await self._connector.connect( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/aiohttp/connector.py", line 544, in connect proto = await self._create_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/aiohttp/connector.py", line 944, in _createconnection , proto = await self._create_direct_connection(req, traces, timeout) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/aiohttp/connector.py", line 1226, in _create_direct_connection transp, proto = await self._wrap_create_connection( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/user/PycharmProjects/Rocky/venv/lib/python3.11/site-packages/aiohttp/connector.py", line 1025, in _wrap_create_connection return await self._loop.create_connection(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 1069, in create_connection sock = await self._connect_sock( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 973, in _connect_sock await self.sock_connect(sock, address) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/selector_events.py", line 632, in sock_connect self._sock_connect(fut, sock, address) File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/selector_events.py", line 649, in _sock_connect handle = self._add_writer( ^^^^^^^^^^^^^^^^^ File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/selector_events.py", line 299, in _add_writer self._check_closed() File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 519, in _check_closed raise RuntimeError('Event loop is closed') RuntimeError: Event loop is closed

**Expected behavior**
should be able to run async tests, including those involving calling APIs (fastapi)

[xiangyan99]

Thanks for the feedback, we’ll investigate asap.

[YalinLi0312]

Hi @yovelcohen , can you share more about how you created srt_table_client and call an operation from it? You can also try to resolve the issue by following our async samples(link). Additionally, I notice the parameter you passed to await srt_table_client.get_entity(1) is in wrong type, we expect the first positional parameter(partition_key) in type string.

In the first code snippet, why are you creating a ClientSesstion object? Actually you can either use the context manager to confirm completion, or call close().

Thanks

[github-actions[bot]]

Hi @yovelcohen. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

[yovelcohen]

@YalinLi0312 Thanks for the response, so first of all, the srt_table_client is just a thin biz logic layer we have above the SDK client, here's some of it's implementation:

class BaseTableManager:
    model_class: TableModelType | None = None
    unique_key: str | None = None
    DEFAULT_TABLE_NAME: str | None = None

    def __init__(self, project: Project | None, table_name: str | None = None):
        self.project = project
        table_name = table_name or self.DEFAULT_TABLE_NAME
        self._client = AzureTableStorageClient.get_client(table_name=table_name)

    @property
    def partition_key(self) -> str:
        return f"{str(self.project.client_id)}_{str(self.project.id)}"

    @property
    def client(self) -> AsyncTableClient:
        return self._client

    async def get_entity(self, row_index: RowIndexType) -> TableModelType:
        try:
            entity = await self._client.get_entity(partition_key=self.partition_key, row_key=str(row_index))
            return self.db_entity_to_model_entity(entity)
        except ResourceNotFoundError as e:
            raise RowNotFoundError(f"Row with index {row_index} not found", row_index=row_index,
                                   project_id=self.project.id) from e

So it's just a wrapper, and the usage of get_entity is actually as expected. Just to reiterate, the behaviour I'm describing only happens in test cases, If I were to call the exact same API with exact same parameters but via curl/postman... it works perfectly fine.

The use of ClientSession when initiating the table client is not something we do regularly, it was for experimenting with how to solve those Event Loop Closed errors, but it didn't help either.

Please let me know if any more questions arise and thanks again :)

[YalinLi0312]

Hi @yovelcohen Thanks for your reply. After we investigate, the issue is not from our SDK as we never touch the loop in SDK. You can check if the loop was closed before running the operation in your test.

[github-actions[bot]]

Hi @yovelcohen. Thank you for opening this issue and giving us the opportunity to assist. We believe that this has been addressed. If you feel that further discussion is needed, please add a comment with the text "/unresolve" to remove the "issue-addressed" label and continue the conversation.

[github-actions[bot]]

Hi @yovelcohen, since you haven’t asked that we /unresolve the issue, we’ll close this out. If you believe further discussion is needed, please add a comment /unresolve to reopen the issue.