search

bricks-cloud / BricksLLM

🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
https://trybricks.ai/
MIT License
901 stars 61 forks source link

Add monthly rate limits #26

Closed spikelu2016 closed 10 months ago

spikelu2016 commented 10 months ago

https://community.openai.com/t/openai-api-proxy-to-share-limited-token-budget-with-others/535403/14

spikelu2016 commented 10 months ago

@LinqLover we support monthly spend limit now

LinqLover commented 10 months ago

Very nice! Would this also work with minutes? Also, monthly cost limits are not yet combinable with hourly cost limits, are they?

spikelu2016 commented 10 months ago

Very nice! Would this also work with minutes? Also, monthly cost limits are not yet combinable with hourly cost limits, are they?

To answer your first question, minutes is not available yet but I can make it available quite easily. For the second question, they are not combinable atm.

LinqLover commented 10 months ago

The first one would be great! The second one, too, but not so urgent. :-)

spikelu2016 commented 10 months ago

The first one would be great! The second one, too, but not so urgent. :-)

Updated. minute sped limit should be supported in the latest version