[Reliability Fix] Anthropic / Bedrock HTTPX - Cache Async Httpx client

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

https://docs.litellm.ai/docs/

Other

12.5k stars 1.45k forks source link

[Reliability Fix] Anthropic / Bedrock HTTPX - Cache Async Httpx client #4211

Closed ishaan-jaff closed 3 months ago

ishaan-jaff commented 3 months ago

Title

Doing 20 RPS

Relevant issues

Type

🆕 New Feature 🐛 Bug Fix 🧹 Refactoring 📖 Documentation 🚄 Infrastructure ✅ Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

vercel[bot] commented 3 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 15, 2024 4:39am