Azure azure-openai-benchmark issues

Azure / azure-openai-benchmark

Azure OpenAI benchmarking tool

MIT License

118 stars 51 forks source link

issues

Newest

Newest Most commented Recently updated Oldest Least commented Least recently updated

Issue while running benchmarking tool in setting the rpm

#55 rajm3180 opened 1 month ago
1
Issue with azure benchmarking tool on passing input tokens greater than 30k

#54 rajm3180 opened 1 month ago
1
Difference in the e2e_avg latency observed in the benchmarking tool and Azure portal

#53 rajm3180 opened 1 month ago
1
Correct poor English

#52 tyler-suard-parker opened 3 months ago
0
Add disclaimer for the max PTU this tool can test

#51 yshahin closed 6 months ago
0
Add tpr stats output, and periodic warning when gen_tpr < 90% of max_tokens

#50 yshahin opened 6 months ago
0
Decoupling retry logics

#49 sushaanttb opened 6 months ago
0
Added handling considering retry-after header

#48 sushaanttb opened 6 months ago
1
Typo fix for presence_penalty and frequency_penalty

#47 sushaanttb closed 6 months ago
0
Decoupling retry logics

#46 sushaanttb closed 6 months ago
0
Added handling using retry-after header

#45 sushaanttb closed 6 months ago
0
Typo fix for presence penalty key and frequency penalty

#44 sushaanttb closed 6 months ago
0
Fix argument "shape" to "shape-profile"

#43 alexmanie opened 6 months ago
0
Purpose of stats call condition in while loop?

#42 sushaanttb opened 6 months ago
0
Properly reflecting the count of "requests"

#41 sushaanttb opened 6 months ago
0
Count of Completed Requests & Total Requests is same

#40 sushaanttb opened 6 months ago
0
Typo fix for presence_penalty arg and adding retry_after_ms header

#39 sushaanttb closed 6 months ago
0
Documenting about the two types of retry logics for 429 errors.

#38 sushaanttb opened 6 months ago
2
Documenting about MAX_RETRY_SECONDS

#37 sushaanttb opened 6 months ago
2
Value for MAX_RETRY_SECONDS should not be hardcoded

#36 sushaanttb opened 6 months ago
2
Fixing the while loop logic of retry mechanism for 429 requests

#35 sushaanttb closed 6 months ago
1
Decoupling Throttling retries logic & Throttling backoff logic.

#34 sushaanttb opened 6 months ago
4
RETRY_AFTER_MS_HEADER not getting properly "honored"

#33 sushaanttb opened 6 months ago
1
Documenting reference for header values used.

#32 sushaanttb opened 6 months ago
0
Considering "retry-after" header as well when retrying throttled requests.

#31 sushaanttb opened 6 months ago
0
Renaming "frequence_penalty" function argument to "frequency_penalty"

#30 sushaanttb opened 6 months ago
0
RequestBuilder is prepared with hardcoded value of "gpt-4-0613"

#29 sushaanttb opened 6 months ago
0
Typo when setting "presence penalty" parameter

#28 sushaanttb opened 6 months ago
0
removed duplicated word

#27 timschps closed 6 months ago
0
Add dynamic aggregation window

#26 michaeltremeer closed 6 months ago
0
Utilization Statistics - avg, P95

#25 edjez opened 7 months ago
0
Benchmark tool Output to CSV

#24 edjez opened 7 months ago
2
Any plan to adding visualization result in this package?

#23 guming3d opened 7 months ago
3
Is it possible to do benchmark testing on APIM which has couple of Azure Open AI resources in backend?

#22 deeepakmhaskar opened 7 months ago
1
Fix retry logic

#21 michaeltremeer closed 6 months ago
2
Hidden/silent effects of MAX_RETRY_SECONDS

#20 michaeltremeer closed 6 months ago
2
Adding feature to support using Custom prompt data instead of random words

#19 guming3d closed 7 months ago
1
Add prevent-server-caching arg

#18 michaeltremeer closed 4 months ago
1
Add 'replay' context generation method

#17 michaeltremeer closed 4 months ago
1
APIM PermissionDenied

#16 jefffeng-ai closed 7 months ago
2
Add counts of requests currently processing

#15 michaeltremeer closed 8 months ago
0
Add tpr stats output, and periodic warning when gen_tpr < 90% of max_tokens

#14 michaeltremeer closed 6 months ago
6
Add saving of logs to disk & combining into CSV

#13 michaeltremeer opened 8 months ago
2
Emit a warning when request e2e latency > aggregation-window

#12 michaeltremeer closed 8 months ago
1
Change prompt to allow for longer generations

#11 technicianted closed 8 months ago
1
Unable to extract values for util_avg and util_95th parameters using the tool

#10 suryakurapati closed 7 months ago
2
Question about using wonderwords to generate random prompt info

#9 guming3d opened 9 months ago
3
Set unlimited TCP connections for aiohttp

#8 technicianted closed 10 months ago
0
Merge context and generation TPM

#7 technicianted closed 10 months ago
0
Add output timestamp

#6 technicianted closed 10 months ago
0