nkowaokwu / igbo_api

An API exposing Igbo words, definitions, and more
https://igboapi.com
Apache License 2.0
316 stars 141 forks source link

Serverless rate limiting for AI model endpoints #814

Open ebubae opened 1 month ago

ebubae commented 1 month ago

Is your feature request related to a problem? Please describe. Existing rate limiting solution requires endpoint to be continually running to correctly limit requests but our serverless infrastructure makes it such that it cannot easily maintain state between teardowns of the application on subsequent requests.

Describe the solution you'd like Use Upstash rate limiting to rate limit incoming requests. More specifically we should

Describe alternatives you've considered Existing Redis solution doesn't scale as well as a serverless solution and does not have built-in rate limiting functionality.