search

Azure / azure-openai-benchmark

Azure OpenAI benchmarking tool
MIT License
130 stars 54 forks source link

Is it possible to do benchmark testing on APIM which has couple of Azure Open AI resources in backend? #22

Open deeepakmhaskar opened 10 months ago

deeepakmhaskar commented 10 months ago

I am trying to do benchmark test on APIM which has couple of Azure Open AI resources in backend. I am getting failures while running the load test.

Can you please suggest? Any suggestions / Feedback welcome.

DavidLiCIG commented 10 months ago

Hi, you might need to override the authentication header to make it work with your APIM instance: https://github.com/Azure/azure-openai-benchmark/blob/84c05e1d0ee59d75db0e319eb8450b06e1431027/benchmark/oairequester.py#L86

That is assuming that the path is still the same. For example, https://{customdomain}.openai.azure.com/openai/deployments/{deploymentName}/chat/completions -> https://{your-apim-domain}/openai/deployments/{deploymentName}/chat/completions.