Add per-minute rate limits

stuartleeks / aoai-simulated-api

An exploration into creating a simulated API implementation for Azure OpenAI (AOAI)

MIT License

11 stars 4 forks source link

Closed stuartleeks closed 2 months ago

stuartleeks commented 2 months ago

Add per-minute rate limits (works towards #17)
Add x-ratelimit-remaining-tokens/x-ratelimit-remaining-requests headers to responses
skip first/last minute in mean token validation