🔒 Enterprise-grade API gateway that helps you monitor and impose cost or rate limits per API key. Get fine-grained access control and monitoring per user, application, or environment. Supports OpenAI, Azure OpenAI, Anthropic, vLLM, and open-source LLMs.
Can the failover feature only be supported by OpenAI and Azure? Can't we support VLLM suppliers?
Is there a load balancing function between two identical LLMs? So, how should I configure it? I can't find the configuration method
Can the failover feature only be supported by OpenAI and Azure? Can't we support VLLM suppliers? Is there a load balancing function between two identical LLMs? So, how should I configure it? I can't find the configuration method