VertexAI/Gemini: Calculate cost based on context window

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]

https://docs.litellm.ai/docs/

Other

12.5k stars 1.45k forks source link

VertexAI/Gemini: Calculate cost based on context window #4245

Closed krrishdholakia closed 3 months ago

krrishdholakia commented 3 months ago

Title

VertexAI/Gemini: Calculate cost based on context window

Relevant issues

Builds on https://github.com/BerriAI/litellm/issues/4229

Type

🆕 New Feature 🧹 Refactoring

Changes

adds cost calculation based on context window for gemini/vertex ai calls

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

new test_gemini_completion_cost in test_completion_cost.py, covers both providers

cc: @gzamboni

vercel[bot] commented 3 months ago

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 17, 2024 7:42pm