Azure / azure-openai-benchmark

Azure OpenAI benchmarking tool
MIT License
130 stars 54 forks source link

Emit a warning when request e2e latency > aggregation-window #12

Closed michaeltremeer closed 11 months ago

michaeltremeer commented 11 months ago

This adds a warning for when requests complete in a time period longer than the aggregation window. If this is happening consistently, these requests will immediately drop out of the statistics for the run, which can result in no windowed statistics (except for the second in which each request returns a response).

Example of issue: image

With warning: image

michaeltremeer commented 11 months ago

No problem, all done!