search

BerriAI / litellm

Python SDK, Proxy Server (LLM Gateway) to call 100+ LLM APIs in OpenAI format - [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, Replicate, Groq]
https://docs.litellm.ai/docs/
Other
14.01k stars 1.66k forks source link

[Feature]: ai21/jamba-1.5 from Google Vertex AI #5399

Closed jguokion closed 2 months ago

jguokion commented 2 months ago

The Feature

I would like to use Jamba 1.5 Model Family from vertex ai.

Jamba 1.5 Mini: AI21’s most efficient and lightweight model, engineered for speed and efficiency in tasks including customer support, document summarization, and text generation.

Jamba 1.5 Large: AI21’s most advanced and largest model that can handle advanced reasoning tasks — such as financial analysis — with exceptional speed and efficiency.

Motivation, pitch

The models are very efficient and have novel Mamba architecture. This is appealing to me.

Twitter / LinkedIn details

No response

traderpedroso commented 2 months ago

Have you test it using custom model - vertex_ai-language-models/jamba-1.5

krrishdholakia commented 2 months ago

Supported with this PR - https://github.com/BerriAI/litellm/pull/5391

Will be live in today's release