issues
search
stuartleeks
/
aoai-simulated-api
An exploration into creating a simulated API implementation for Azure OpenAI (AOAI)
MIT License
11
stars
4
forks
source link
Finish rate-limiting implementation for OpenAI
#17
Closed
stuartleeks
closed
1 month ago
stuartleeks
commented
4 months ago
Need to ensure that rate-limits are applied on requests + tokens over both 10s and 1min intervals (with an overflow of a single request)
Allow a single burst over the rate-limits (TODO determine which period this applies to)