fix(token_counter.py): New `get_modified_max_tokens' helper func

[krrishdholakia]

Title

New `get_modified_max_tokens' helper func

Returns the user's max output tokens, adjusted for:

• the size of input - for models where input + output can't exceed X
• model max output tokens - for models where there is a separate output token limit

Relevant issues

Fixes https://github.com/BerriAI/litellm/issues/4439

Type

🆕 New Feature 🐛 Bug Fix 🧹 Refactoring

Changes

• refactors the modify_max_token logic in wrapper to a separate helper
• fixes the helper func logic

[REQUIRED] Testing - Attach a screenshot of any new tests passing locall

If UI changes, send a screenshot/GIF of working UI fixes

new test in test_token_counter.py

[vercel[bot]]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
litellm	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Jun 27, 2024 10:43pm