search

Azure / azure-openai-benchmark

Azure OpenAI benchmarking tool
MIT License
128 stars 54 forks source link

Is it possible to do benchmark testing on APIM which has couple of Azure Open AI resources in backend? #22

Open deeepakmhaskar opened 9 months ago

deeepakmhaskar commented 9 months ago

I am trying to do benchmark test on APIM which has couple of Azure Open AI resources in backend. I am getting failures while running the load test.

Can you please suggest? Any suggestions / Feedback welcome.

DavidLiCIG commented 9 months ago

Hi, you might need to override the authentication header to make it work with your APIM instance: https://github.com/Azure/azure-openai-benchmark/blob/84c05e1d0ee59d75db0e319eb8450b06e1431027/benchmark/oairequester.py#L86

That is assuming that the path is still the same. For example, https://{customdomain}.openai.azure.com/openai/deployments/{deploymentName}/chat/completions -> https://{your-apim-domain}/openai/deployments/{deploymentName}/chat/completions.