microsoft / azure-openai-service-proxy

The Azure AI proxy service facilitates easy access to Azure AI resources for workshops and hackathons. It offers a Playground-like interface and supports Azure AI SDKs. Access is granted through a time-limited API key and endpoint.
https://microsoft.github.io/azure-openai-service-proxy/
MIT License
58 stars 34 forks source link

Move rate limiting out of auth pipeline #311

Open aaronpowell opened 2 months ago

aaronpowell commented 2 months ago

The auth pipeline does two things on token validation, first is makes sure the token exists is valid for the event, the second is to implement a rate limiting check.

But rate limiting (exceeding the "max request cap") is not really an auth stage, it should be done using the rate limiting middleware in asp.net. This will allow us to properly return a 429 status code.