search

Azure-Samples / azure-search-openai-demo

A sample app for the Retrieval-Augmented Generation pattern running in Azure, using Azure AI Search for retrieval and Azure OpenAI large language models to power ChatGPT-style and Q&A experiences.
https://azure.microsoft.com/products/search
MIT License
5.9k stars 4.04k forks source link

APIError Invalid response object from API: '{ "statusCode": 401, "message": "Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired." }' (HTTP response code was 401))> #458

Open chip-davis opened 1 year ago

chip-davis commented 1 year ago

This issue is for a: (mark with an x)

- [X ] bug report -> please search issues before submitting
- [ ] feature request
- [ ] documentation issue or request
- [ ] regression (a behavior that used to work and stopped in a new release)

Minimal steps to reproduce

After ingesting a large amount of documents via prep docs (100ish) I receive this error. This error occurs after being rate limited. It will usually recover successfully but after a second round of rate limiting, I run into this error. I have removed max retries and adjusted the timeout in the AzureDeveloperCliCredential call but that does not seem to help.

Any log messages given by the failure

APIError Invalid response object from API: '{ "statusCode": 401, "message": "Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired." }' (HTTP response code was 401))>

Expected/desired behavior

Expected behavior would be for the program to recover after rate limit and not lose credentials.

OS and Version?

Windows 7, 8 or 10. Linux (which distribution). macOS (Yosemite? El Capitan? Sierra?)

Windows 11

azd version?

run azd version and copy paste here.

azd version 1.1.0 (commit ea9cb12575734ee6a5f99c4d415c1a51d6f32d3e)

Versions

Using latest commit from repo.

Mention any other details that might be useful

Any guidance would be appreciated!

davidwboyd commented 1 year ago

I am experiencing this same issue

Pked01 commented 1 year ago

me and my colleagues are experiencing similar issue

pamelafox commented 1 year ago

This is basically the same issue as https://github.com/Azure-Samples/azure-search-openai-demo/issues/431 Please see that issue for some suggestions

yak18t commented 1 year ago

Same for me. I try large PDF with 100+ pages

pamelafox commented 1 year ago

We don't have an elegant solution yet, but I think a workaround is to figure out when your token is expiring, and put code in to regenerate the token, such as is done here:

https://github.com/Azure-Samples/azure-search-openai-demo/blob/52abf79fdf545fc29c5c9159b88a6ff9010f4ed2/app/backend/app.py#L128

chip-davis commented 1 year ago

We don't have an elegant solution yet, but I think a workaround is to figure out when your token is expiring, and put code in to regenerate the token, such as is done here:

https://github.com/Azure-Samples/azure-search-openai-demo/blob/52abf79fdf545fc29c5c9159b88a6ff9010f4ed2/app/backend/app.py#L128

Thanks for the idea! I will try to implement this on my end.

pamelafox commented 1 year ago

A fix has been merged for this error. The fix refreshes the token every 5 minute. Please re-open if you still encounter it during prepdocs.py.

Note that you will encounter it in production if you have more users than your TPM allows. We default the TPM to 30 in main.bicep, but you likely want to increase to the max if deploying for production.

rubikron commented 9 months ago

Hi there, I am still getting this issue. I am trying to upload abt 20,000 documents (each less than a page) and after 15-16 minutes or so this error pops up. This is with the demo version from December 15th, 2023

stbere commented 7 months ago

Errors out after about an hour for me. PDFs with several pages, then timed-out! So i'm having to re-run prepdocs.py to capture my entire library I'm needing to upload (+1600 docs total).

pamelafox commented 7 months ago

Can you share the traceback you see? I wonder if it's a different token timing out.

stbere commented 7 months ago

Can you share the traceback you see? I wonder if it's a different token timing out.

Absolutely, here's what I get below. It seems to have happened hourly. I hit 'azd auth login' before re-running prepdocs.py to make sure i'm authenticated 

Traceback (most recent call last):
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocs.py", line 256, in <module>
    loop.run_until_complete(main(file_strategy, azd_credential, args))
  File "C:\Users\steve41320sa\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocs.py", line 131, in main
    await strategy.run(search_info)
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\filestrategy.py", line 63, in run
    await search_manager.update_content(sections)
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\searchmanager.py", line 150, in update_content
    embeddings = await self.embeddings.create_embeddings(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\embeddings.py", line 116, in create_embeddings
    return await self.create_embedding_batch(texts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\embeddings.py", line 87, in create_embedding_batch
    async for attempt in AsyncRetrying(
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\tenacity\_asyncio.py", line 71, in __anext__
    do = self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\tenacity\__init__.py", line 314, in iter
    return fut.result()
           ^^^^^^^^^^^^
  File "C:\Users\steve41320sa\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\steve41320sa\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\embeddings.py", line 94, in create_embedding_batch
    emb_response = await client.embeddings.create(model=self.open_ai_model_name, input=batch.texts)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\resources\embeddings.py", line 198, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\_base_client.py", line 1542, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\_base_client.py", line 1316, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\_base_client.py", line 1368, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.'}
Rishabh7121999 commented 6 months ago

Can you share the traceback you see? I wonder if it's a different token timing out.

Absolutely, here's what I get below. It seems to have happened hourly. I hit 'azd auth login' before re-running prepdocs.py to make sure i'm authenticated

Traceback (most recent call last):
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocs.py", line 256, in <module>
    loop.run_until_complete(main(file_strategy, azd_credential, args))
  File "C:\Users\steve41320sa\AppData\Local\Programs\Python\Python311\Lib\asyncio\base_events.py", line 653, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocs.py", line 131, in main
    await strategy.run(search_info)
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\filestrategy.py", line 63, in run
    await search_manager.update_content(sections)
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\searchmanager.py", line 150, in update_content
    embeddings = await self.embeddings.create_embeddings(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\embeddings.py", line 116, in create_embeddings
    return await self.create_embedding_batch(texts)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\embeddings.py", line 87, in create_embedding_batch
    async for attempt in AsyncRetrying(
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\tenacity\_asyncio.py", line 71, in __anext__
    do = self.iter(retry_state=self._retry_state)
         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\tenacity\__init__.py", line 314, in iter
    return fut.result()
           ^^^^^^^^^^^^
  File "C:\Users\steve41320sa\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 449, in result
    return self.__get_result()
           ^^^^^^^^^^^^^^^^^^^
  File "C:\Users\steve41320sa\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures\_base.py", line 401, in __get_result
    raise self._exception
  File "C:\AI_Project\FlexLibraryV2\scripts\prepdocslib\embeddings.py", line 94, in create_embedding_batch
    emb_response = await client.embeddings.create(model=self.open_ai_model_name, input=batch.texts)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\resources\embeddings.py", line 198, in create
    return await self._post(
           ^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\_base_client.py", line 1542, in post
    return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\_base_client.py", line 1316, in request
    return await self._request(
           ^^^^^^^^^^^^^^^^^^^^
  File "C:\AI_Project\FlexLibraryV2\scripts\.venv\Lib\site-packages\openai\_base_client.py", line 1368, in _request
    raise self._make_status_error_from_response(err.response) from None
openai.AuthenticationError: Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.'}

Hello @stbere, were you able to resolve the error? I'm getting similar kind of error, while using AzureOpenAI Embeddings while creating a vectorstore in CosmosDB.

`vectorstore_page= AzureCosmosDBVectorSearch.from_documents( loaded_doc, azure_embeddings, collection=collection_page, index_name=INDEX_NAME, )

AuthenticationError: Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com/), or have expired.'}`

mikedizon commented 4 months ago

@pamelafox here's the traceback i'm getting. the pdf i am trying to index is 7000 pages. 

Traceback (most recent call last):
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/async_base_polling.py", line 89, in run
    await self._poll()
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/async_base_polling.py", line 118, in _poll
    await self.update_status()
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/async_base_polling.py", line 141, in update_status
    _raise_if_bad_http_status_and_method(self._pipeline_response.http_response)
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/base_polling.py", line 156, in _raise_if_bad_http_status_and_method
    raise BadStatus("Invalid return status {!r} for {!r} operation".format(code, response.request.method))
azure.core.polling.base_polling.BadStatus: Invalid return status 401 for 'GET' operation

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/Users/xxx/Code/azure-search-openai-demo/./app/backend/prepdocs.py", line 494, in <module>
    loop.run_until_complete(main(ingestion_strategy, setup_index=not args.remove and not args.removeall))
  File "/opt/homebrew/Cellar/python@3.11/3.11.9/Frameworks/Python.framework/Versions/3.11/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
    return future.result()
           ^^^^^^^^^^^^^^^
  File "/Users/xxx/Code/azure-search-openai-demo/./app/backend/prepdocs.py", line 225, in main
    await strategy.run()
  File "/Users/xxx/Code/azure-search-openai-demo/app/backend/prepdocslib/filestrategy.py", line 84, in run
    sections = await parse_file(file, self.file_processors, self.category, self.image_embeddings)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Code/azure-search-openai-demo/app/backend/prepdocslib/filestrategy.py", line 26, in parse_file
    pages = [page async for page in processor.parser.parse(content=file.content)]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Code/azure-search-openai-demo/app/backend/prepdocslib/filestrategy.py", line 26, in <listcomp>
    pages = [page async for page in processor.parser.parse(content=file.content)]
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Code/azure-search-openai-demo/app/backend/prepdocslib/pdfparser.py", line 57, in parse
    form_recognizer_results = await poller.result()
                              ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/_async_poller.py", line 179, in result
    await self.wait()
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/_async_poller.py", line 191, in wait
    await self._polling_method.run()
  File "/Users/xxx/Code/azure-search-openai-demo/.venv/lib/python3.11/site-packages/azure/core/polling/async_base_polling.py", line 93, in run
    raise HttpResponseError(response=self._pipeline_response.http_response, error=err) from err
azure.core.exceptions.HttpResponseError: (None) Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.
Code: None
Message: Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.
SHDeniz commented 1 week ago

Same here:

Exception has occurred: AuthenticationError       (note: full exception trace is shown but execution is paused at: <module>)
Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.'}
httpx.HTTPStatusError: Client error '401 Unauthorized' for url 'https://cog-733ownwusqico.openai.azure.com/openai/deployments/text-embedding-3-large/embeddings?api-version=2023-05-15'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/401

During handling of the above exception, another exception occurred:

  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocslib\embeddings.py", line 105, in create_embedding_batch
    async for attempt in AsyncRetrying(
  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocslib\embeddings.py", line 105, in create_embedding_batch
    async for attempt in AsyncRetrying(
  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocslib\embeddings.py", line 149, in create_embeddings
    return await self.create_embedding_batch(texts, dimensions_args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocslib\searchmanager.py", line 251, in update_content
    embeddings = await self.embeddings.create_embeddings(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocslib\filestrategy.py", line 93, in run
    await search_manager.update_content(sections, blob_image_embeddings, url=file.url)
  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocs.py", line 221, in main
    await strategy.run()
  File "C:\Users\Server\OneDrive - Serverhero GmbH\Dokumente\Code\Chatbots\demoApp2\app\backend\prepdocs.py", line 485, in <module> (Current frame)
    loop.run_until_complete(main(ingestion_strategy, setup_index=not args.remove and not args.removeall))
openai.AuthenticationError: Error code: 401 - {'statusCode': 401, 'message': 'Unauthorized. Access token is missing, invalid, audience is incorrect (https://cognitiveservices.azure.com), or have expired.'}
pamelafox commented 6 days ago

@SHDeniz Can you check if you have an environment variable of OPENAI_API_KEY ? I helped someone yesterday whose environment was getting confused by that global variable, since the app should be using keyless authentication by default.

cforce commented 2 days ago

It seems (by debugging) even if OPENAI_HOST=azure - this is ignored if "AZURE_OPENAI_API_KEY_OVERRIDE" is that overulkes to choose
openai_client = AsyncAzureOpenAI(api_version=api_version, azure_endpoint=endpoint, api_key=api_key)

Issue is - if then AZURE_OPENAI_CUSTOM_URL is not set -- it seems to fail with the 401(no access token)

See my with logs extended code :

2024-09-17T16:33:20.585108837Z 2024-09-17 16:33:20,584 - INFO - OPENAI_HOST is azure, setting up Azure OpenAI client 2024-09-17T16:33:20.588423888Z 2024-09-17 16:33:20,586 - INFO - USE_USER_UPLOAD is true, setting up user upload feature 2024-09-17T16:33:20.588443788Z 2024-09-17 16:33:20,586 - INFO - AZURE_OPENAI_API_KEY_OVERRIDE found, using as api_key for Azure OpenAI client 2024-09-17T16:33:20.590638922Z 2024-09-17 16:33:20,590 - INFO - USE_SPEECH_OUTPUT_AZURE is true, setting up Azure speech service 2024-09-17T16:33:20.592438850Z 2024-09-17 16:33:20,591 - INFO - OPENAI_HOST is azure, setting up Azure OpenAI client 2024-09-17T16:33:20.593700569Z 2024-09-17 16:33:20,592 - INFO - OPENAI_HOST is azure, setting up Azure OpenAI client 2024-09-17T16:33:20.596148007Z 2024-09-17 16:33:20,594 - INFO - AZURE_OPENAI_API_KEY_OVERRIDE found, using as api_key for Azure OpenAI client 2024-09-17T16:33:20.596167507Z 2024-09-17 16:33:20,594 - INFO - AZURE_OPENAI_API_KEY_OVERRIDE found, using as api_key for Azure OpenAI client

pamelafox commented 2 days ago

So you are trying to use both a key and AZURE_OPENAI_CUSTOM_URL ?

cforce commented 2 days ago

Yes. We use azure OpenAI model as a shared service by IT using their host and a key we got to auth . It seems like even if I set the AZURE_OPENAI_CUSTOM_URL to that external OpenAI api, the error is still the same . Is there something wrong with auth scopes?

cforce commented 22 hours ago

@SHDeniz Can you check if you have an environment variable of OPENAI_API_KEY ? I helped someone yesterday whose environment was getting confused by that global variable, since the app should be using keyless authentication by default.

I fixed it by adding this to the prepdocs.sh

integratedVectorizationArg="" if [ -n "$USE_FEATURE_INT_VECTORIZATION" ]; then integratedVectorizationArg="--useintvectorization $USE_FEATURE_INT_VECTORIZATION" fi

if [ -n "$AZURE_OPENAI_API_KEY_OVERRIDE" ]; then openAiApiKeyArg="--openaikey $AZURE_OPENAI_API_KEY_OVERRIDE" elif [ -n "$OPENAI_API_KEY" ]; then openAiApiKeyArg="--openaikey $OPENAI_API_KEY" else openAiApiKeyArg="" fi