search

microsoftgraph / msgraph-sdk-python

MIT License
360 stars 50 forks source link

ItemsRequestBuilder.request_adapter.send_async raises AttributeError: 'NoneType' object has no attribute '__aenter__' #645

Open nbtk123 opened 6 months ago

nbtk123 commented 6 months ago

Hello,

I have a scenario, where I use GraphServiceClient based on ClientSecretCredential to paginate over some site-list-items resources, and after an hour or so, the aiohttp session is becoming None.

Excerpt from the exception:

### Beginning of stack trace ###

[Some previous code of mine...]
[Some previous code of SDK...]

  File "/src/pkgs/azure/identity/aio/_credentials/client_secret.py", line 71, in _request_token
    return await self._client.obtain_token_by_client_secret(scopes, self._secret, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
...
...

  File "/src/pkgs/azure/core/pipeline/transport/_aiohttp.py", line 127, in open
    await self.session.__aenter__()
          ^^^^^^^^^^^^^^^^^^^^^^^

AttributeError: 'NoneType' object has no attribute '__aenter__'"

### End of stack trace ###

The pagination is done inside a loop, using the odata_next_link which is returned with the response, thus requesting the next page.

Eventually, after about an hour, the SDK tries to renew the token but fails with the exception, described further below.

I dig deep into the source code to detect if I did something wrong, but I didn't find anything. I have a few thoughts but I don't want to confuse you guys or mislead, so I'll just describe my code and the full exception stack trace.

My setup

Python versoin: 3.11.5

msgraph version, from pdm lock file:

[[package]]
name = "msgraph-sdk"
version = "1.2.0"
requires_python = ">=3.8"
summary = "The Microsoft Graph Python SDK"
dependencies = [
    "azure-identity>=1.12.0",
    "microsoft-kiota-abstractions<2.0.0,>=1.0.0",
    "microsoft-kiota-authentication-azure<2.0.0,>=1.0.0",
    "microsoft-kiota-http<2.0.0,>=1.0.0",
    "microsoft-kiota-serialization-form>=0.1.0",
    "microsoft-kiota-serialization-json<2.0.0,>=1.0.0",
    "microsoft-kiota-serialization-multipart>=0.1.0",
    "microsoft-kiota-serialization-text<2.0.0,>=1.0.0",
    "msgraph-core>=1.0.0",
]
files = [
    {file = "msgraph-sdk-1.2.0.tar.gz", hash = "sha256:689eec74fcb5cb29446947e4761fa57edeeb3ec1dccd7975c44d12d8d9db9c4f"},
    {file = "msgraph_sdk-1.2.0-py3-none-any.whl", hash = "sha256:4a9f706413c0a497cdfffd0b741122a5e73206333d566d115089cef9f4adadb7"},
]

My code flow

My code flow, in short, is as such:

# In some consts file
ERROR_MAPPING = {
    "4XX": ODataError,
    "5XX": ODataError,
}
ALL_ADMIN_SITES_LIST = "DO_NOT_DELETE_SPLIST_TENANTADMIN_ALL_SITES_AGGREGATED_SITECOLLECTIONS"

credential = ClientSecretCredential(some_tenant_id, some_app_id, some_app_secret)
graph_client = GraphServiceClient(credentials=credential, scopes=some_scopes)

def sites_generator():
    while is_first_run or next_url is not None:
        is_first_run = False

        request_builder = graph_client.sites.by_site_id(SOME_SITE_ID).lists.by_list_id(ALL_ADMIN_SITES_LIST).items

        request_information = request_builder.to_get_request_information()
        if next_url is not None:
            request_information.url = next_url

        response = await request_builder.request_adapter.send_async(request_information, SiteCollectionResponse, ERROR_MAPPING)

        # will be None once there are no further pages
        next_url = page.odata_next_link
        yield response

async for item in sites_generator():
    try:
        # do something with response
    except Exception:
        # On any exception, we continue to loop!

### End of my code ###

The exception stack trace

And the exception I receive, seems to be during the token "refresh". I double-quote "refresh" because the SDK doesn't use the refresh token but just requests a new one using the obtain_token_by_client_secret function.

Traceback (most recent call last):

[Some previous code of mine...]

  File "/src/pkgs/common/services/o365/msgraph/paginator/paginator.py", line 30, in get_page
    await request_builder.request_adapter.send_async(request_information, collection_response_type, ERROR_MAPPING)
  File "/src/pkgs/kiota_http/httpx_request_adapter.py", line 178, in send_async
    response = await self.get_http_response_message(request_info, parent_span)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/kiota_http/httpx_request_adapter.py", line 523, in get_http_response_message
    await self._authentication_provider.authenticate_request(
  File "/src/pkgs/kiota_abstractions/authentication/base_bearer_token_authentication_provider.py", line 50, in authenticate_request
    token = await self.access_token_provider.get_authorization_token(
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/kiota_authentication_azure/azure_identity_access_token_provider.py", line 106, in get_authorization_token
    result = await result
             ^^^^^^^^^^^^
  File "/src/pkgs/azure/identity/aio/_internal/get_token_mixin.py", line 86, in get_token
    token = await self._request_token(*scopes, claims=claims, tenant_id=tenant_id, **kwargs)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/identity/aio/_credentials/client_secret.py", line 71, in _request_token
    return await self._client.obtain_token_by_client_secret(scopes, self._secret, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/identity/aio/_internal/aad_client.py", line 48, in obtain_token_by_client_secret
    return await self._run_pipeline(request, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/identity/aio/_internal/aad_client.py", line 84, in _run_pipeline
    response = await self._pipeline.run(request, retry_on_methods=self._POST, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 221, in run
    return await first_node.send(pipeline_request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/src/pkgs/azure/core/pipeline/policies/_retry_async.py", line 179, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 69, in send
    response = await self.next.send(request)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/src/pkgs/azure/core/pipeline/_base_async.py", line 106, in send
    await self._sender.send(request.http_request, **request.context.options),
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/src/pkgs/azure/core/pipeline/transport/_aiohttp.py", line 231, in send
    await self.open()
  File "/src/pkgs/azure/core/pipeline/transport/_aiohttp.py", line 127, in open
    await self.session.__aenter__()
          ^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute '__aenter__'

Would love you help on this!

Thanks!

shemogumbe commented 1 month ago

Hello, this could be due to a limitation on the token lifetime caused by CAE, https://github.com/Azure/azure-sdk-for-python/blob/48cdbb2d3f6422a74faca22a12adb41c26dd475e/sdk/core/azure-core/azure/core/credentials_async.py#L21, To fix this, we have on our side put the flag to true, so tokens can live for upto 24 hours.

Upgrading to the latest release of https://github.com/microsoft/kiota-authentication-azure-python/releases/tag/v1.1.0 should fix this