clearlydefined / operations

Operational documentation and tools
3 stars 10 forks source link

Triage Service exceptions: Azure Search Throttling (/definitions) #28

Open grvillic opened 4 years ago

grvillic commented 4 years ago
requests 
| where resultCode == 500
| join (
 exceptions
 ) on operation_Id 
| extend pathPrefix = extract("/(\\w+)", 0, tostring(operation_Name))
| summarize count() by  outerMessage, pathPrefix
| order by count_  desc 
| where outerMessage contains "throttling" 
grvillic commented 4 years ago

Failed to execute request. Reduce the rate of requests, or adjust the number of replicas/partitions. See http://aka.ms/azure-search-throttling for more information

You can see some of those exceptions here.

After looking at the ~6.4s wait to get the 503, we thought it might be good to just retry before upscaling. Hence code should be updated to retry on 503 in here.