search

microsoft / azurechat

🤖 💼 Azure Chat Solution Accelerator powered by Azure Open AI Service
MIT License
1.15k stars 958 forks source link

Not indexing documents, error #268

Closed rimvel closed 6 months ago

rimvel commented 7 months ago

SCR-20231201-owvf

rimvel commented 7 months ago

Indexes up to 10/140 and this then this error.

jussiroine commented 7 months ago

We're seeing the same, deployment in West Europe. Web App logs reveal nothing, as expected, and seems like this could be a capacity issue on WE for Document Intelligence and/or Azure OpenAI embeddings API.

vtxjholter commented 7 months ago

Also seeing this same issue.

jussiroine commented 7 months ago

Debugging this a bit more, the issue stems from Azure OpenAI rate limits:

"Requests to the Embeddings_Create Operation under Azure OpenAI API version 2023-09-01-preview have exceeded call rate limit of your current OpenAI S0 pricing tier. Please go here: https://aka.ms/oai/quotaincrease if you would like to further increase the default rate limit."

Thus, the Document Intelligence is hitting the Azure OpenAI embeddings API a bit too heavily, and I guess MSFT introduces stricter rate limits for AOAI instances in the past week, or so. Will try to see if the quota increase can be applied.

Update: I updated the rate limit for the embeddings model from the default (10K) to 20K. It didn't flinch, and the issue wasn't resolved. I updated to 240K, and the issue was fixed with this. Seems to take a minute or two to update.

ReginaLin24 commented 6 months ago

Hi @jussiroine, just wondering did you need to redeploy the solution e.g. model deployments after the quota increase? we have raised the quota increase request (and approved) but still seeing the same behaviour.

jussiroine commented 6 months ago

@ReginaLin24 no need to redeploy. Web App restart required, as usual to clear up any existing connectivity/sessions. The quota increase request is not needed - the quota has to be increased from within AOAI instance, and that you can do directly from the AOAI model settings.

000140

rimvel commented 6 months ago

This actually solved the issue. Thank you for the hint!