BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
14.07k stars 1.66k forks source link

(feat) add usage / cost tracking for Anthropic passthrough routes #6835

Closed ishaan-jaff closed 1 day ago

ishaan-jaff commented 1 day ago

Title

Relevant issues

Type

πŸ†• New Feature πŸ› Bug Fix 🧹 Refactoring πŸ“– Documentation πŸš„ Infrastructure βœ… Test

Changes

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

Description by Korbit AI

What change is being made?

Refactor and enhance the Anthropic pass-through routes to include usage and cost tracking, moving response processing logic into a new AnthropicConfig class, and updating logging behavior for improved consistency.

Why are these changes being made?

This change aims to improve the system's clarity and efficiency by modularizing the code and enhancing it with usage/cost tracking capabilities. It addresses the need for consistent transforming and logging of responses from Anthropic routes and adds missing features to support cost management, aligning with existing OpenAI response formats.

Is this description stale? Ask me to generate a new description by commenting /korbit-generate-pr-description

korbit-ai[bot] commented 1 day ago

Hey there, my name is Korbit. I've joined the team as your AI code reviewer. I'll automatically scan your PRs and raise any issues I find. I can also help you write a PR description. As you work with me, I'll learn about your priorities and codebase and get smarter over time. My goal is to help improve your code quality and save time for the human reviewers on your team. Let's get started!

vercel[bot] commented 1 day ago

The latest updates on your projects. Learn more about Vercel for Git β†—οΈŽ

Name Status Preview Comments Updated (UTC)
litellm βœ… Ready (Inspect) Visit Preview πŸ’¬ Add feedback Nov 21, 2024 0:12am