fix: Improve round-robin provider selection and add Claude max_tokens auto-fill

yym68686 / uni-api

This is a project that unifies the management of LLM APIs. It can call multiple backend services through a unified API interface, convert them to the OpenAI format uniformly, and support load balancing. Currently supported backend services include: OpenAI, Anthropic, DeepBricks, OpenRouter, Gemini, Vertex, etc.

323 stars 40 forks source link

fix: Improve round-robin provider selection and add Claude max_tokens auto-fill #3

Closed omegaduncan closed 2 months ago

omegaduncan commented 2 months ago

Fix round-robin selection to ensure proper load distribution
Use class-level tracking for last provider index
Auto-fill max_tokens for Vertex Claude models (8192 for 3-5 series, 4096 for others)