Weird/unexpected error with APIM load balancer

[diberry]

I used this repo to test the ACA LB, which worked. I could see the streaming logs in ACA of LB to see that it worked.

Then I needed to test APIM. I deployed APIM and got the BE URL and set it in this repo and azd up'd again.

It took a really long time to deploy but I didn't see any errors. When I tried the website, it gave me the following screenshot.

I don't know why it thinks I'm going against OpenAI when my env file says Azure OpenAI.

AZURE_AUTH_TENANT_ID=""
AZURE_DOCUMENTINTELLIGENCE_RESOURCE_GROUP="rg-diberry-py-chat-0511-01"
AZURE_DOCUMENTINTELLIGENCE_SERVICE="cog-di-k6bwjkzvzcdoe"
AZURE_ENV_NAME="diberry-py-chat-0511-01"
AZURE_KEY_VAULT_NAME=""
AZURE_LOCATION="eastus2"
AZURE_OPENAI_API_VERSION=""
AZURE_OPENAI_CHATGPT_DEPLOYMENT="chat"
AZURE_OPENAI_CHATGPT_MODEL="gpt-35-turbo"
AZURE_OPENAI_CUSTOM_URL="https://apim-pryjlfx4rfqs6.azure-api.net"
AZURE_OPENAI_EMB_DEPLOYMENT="embedding"
AZURE_OPENAI_EMB_MODEL_NAME="text-embedding-ada-002"
AZURE_OPENAI_GPT4V_DEPLOYMENT="gpt-4v"
AZURE_OPENAI_GPT4V_MODEL="gpt-4"
AZURE_OPENAI_RESOURCE_GROUP="rg-diberry-py-chat-0511-01"
AZURE_OPENAI_SERVICE="cog-k6bwjkzvzcdoe"
AZURE_RESOURCE_GROUP="rg-diberry-py-chat-0511-01"
AZURE_SEARCH_INDEX="gptkbindex"
AZURE_SEARCH_SECRET_NAME=""
AZURE_SEARCH_SEMANTIC_RANKER="free"
AZURE_SEARCH_SERVICE="gptkb-k6bwjkzvzcdoe"
AZURE_SEARCH_SERVICE_ASSIGNED_USERID="c66c97a6-9037-445e-91f2-4f608a4aca7d"
AZURE_SEARCH_SERVICE_RESOURCE_GROUP="rg-diberry-py-chat-0511-01"
AZURE_STORAGE_ACCOUNT="stk6bwjkzvzcdoe"
AZURE_STORAGE_CONTAINER="content"
AZURE_STORAGE_RESOURCE_GROUP="rg-diberry-py-chat-0511-01"
AZURE_SUBSCRIPTION_ID="2123cb08-23fa-4a80-afec-4ec9e015433f"
AZURE_TENANT_ID=""
AZURE_USERSTORAGE_ACCOUNT=""
AZURE_USERSTORAGE_CONTAINER="user-content"
AZURE_USERSTORAGE_RESOURCE_GROUP="rg-diberry-py-chat-0511-01"
AZURE_USE_AUTHENTICATION="false"
AZURE_VISION_ENDPOINT=""
BACKEND_URI="https://app-backend-k6bwjkzvzcdoe.azurewebsites.net"
OPENAI_API_KEY=""
OPENAI_HOST="azure_custom"
OPENAI_ORGANIZATION=""

[pamelafox]

That error could also come from Azure OpenAI, it's the same class for both clients, I believe. Did you find the actual error in the logs / App Insights?

[vrkmajeti]

I also just got the same error. It happens once in a while and it works again. Could you please advice on the fix? Attached the application logs: 2024_06_06_lw0sdlwk000633_default_docker.log

2024-06-06T18:23:35.042919597Z 2024-06-06 18:23:35,038 - root - ERROR - Exception while generating response stream: Error code: 401 - {'error': {'code': 'PermissionDenied', 'message': 'Principal does not have access to API/Operation.'}} 2024-06-06T18:23:35.043445601Z Traceback (most recent call last): 2024-06-06T18:23:35.043456001Z File "/tmp/8dc6ed61a0d73d3/app.py", line 180, in format_as_ndjson 2024-06-06T18:23:35.043460801Z async for event in r: 2024-06-06T18:23:35.043464901Z File "/tmp/8dc6ed61a0d73d3/approaches/chatapproach.py", line 150, in run_with_streaming 2024-06-06T18:23:35.043469001Z extra_info, chat_coroutine = await self.run_until_final_call( 2024-06-06T18:23:35.043473001Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043487201Z File "/tmp/8dc6ed61a0d73d3/approaches/chatreadretrieveread.py", line 151, in run_until_final_call 2024-06-06T18:23:35.043492601Z vectors.append(await self.compute_text_embedding(query_text)) 2024-06-06T18:23:35.043496501Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043500601Z File "/tmp/8dc6ed61a0d73d3/approaches/approach.py", line 231, in compute_text_embedding 2024-06-06T18:23:35.043505001Z embedding = await self.openai_client.embeddings.create( 2024-06-06T18:23:35.043509001Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043513001Z File "/tmp/8dc6ed61a0d73d3/antenv/lib/python3.11/site-packages/opentelemetry/instrumentation/openai/shared/embeddings_wrappers.py", line 91, in aembeddings_wrapper 2024-06-06T18:23:35.043517401Z response = await wrapped(*args, **kwargs) 2024-06-06T18:23:35.043521101Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043524901Z File "/tmp/8dc6ed61a0d73d3/antenv/lib/python3.11/site-packages/openai/resources/embeddings.py", line 214, in create 2024-06-06T18:23:35.043528901Z return await self._post( 2024-06-06T18:23:35.043532701Z ^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043536501Z File "/tmp/8dc6ed61a0d73d3/antenv/lib/python3.11/site-packages/openai/_base_client.py", line 1725, in post 2024-06-06T18:23:35.043540601Z return await self.request(cast_to, opts, stream=stream, stream_cls=stream_cls) 2024-06-06T18:23:35.043555101Z ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043559601Z File "/tmp/8dc6ed61a0d73d3/antenv/lib/python3.11/site-packages/openai/_base_client.py", line 1428, in request 2024-06-06T18:23:35.043563501Z return await self._request( 2024-06-06T18:23:35.043567301Z ^^^^^^^^^^^^^^^^^^^^ 2024-06-06T18:23:35.043571001Z File "/tmp/8dc6ed61a0d73d3/antenv/lib/python3.11/site-packages/openai/_base_client.py", line 1519, in _request 2024-06-06T18:23:35.043577102Z raise self._make_status_error_from_response(err.response) from None 2024-06-06T18:23:35.043581202Z openai.AuthenticationError: Error code: 401 - {'error': {'code': 'PermissionDenied', 'message': 'Principal does not have access to API/Operation.'}}