[Feature]: Support more Google models

04cfb1ed commented 1 week ago

The Feature

I was testing LiteLLM with ALL available models provided by Google, but I found that some of them are not working

Error: Error code: 429 - {'error': {'message': 'No deployments available for selected model, Try again in 60 seconds. Passed model=<the-model-name>. pre-call-checks=False, allowed_model_region=n/a', 'type': 'None', 'param': 'None', 'code': 429}}

Models not working:

vertex_ai/text-bison@002
vertex_ai/text-bison-32k
vertex_ai/text-bison-32k@002
vertex_ai/chat-bison-32k@002
vertex_ai/code-bison@002
vertex_ai/code-bison-32k
vertex_ai/code-bison-32k@002
vertex_ai/code-gecko@latest
vertex_ai/codechat-bison@latest
vertex_ai/codechat-bison@002
vertex_ai/codechat-bison-32k@002
vertex_ai/gemini-1.0-ultra
vertex_ai/gemini-1.0-ultra-001

Some of them are here

Full list of models tested (models that are not in the first list are working fine in a basic testing)

vertex_ai/text-bison
vertex_ai/text-bison@001
vertex_ai/text-bison@002
vertex_ai/text-bison-32k
vertex_ai/text-bison-32k@002
vertex_ai/text-unicorn
vertex_ai/text-unicorn@001
vertex_ai/chat-bison
vertex_ai/chat-bison@001
vertex_ai/chat-bison@002
vertex_ai/chat-bison-32k
vertex_ai/chat-bison-32k@002
vertex_ai/code-bison
vertex_ai/code-bison@001
vertex_ai/code-bison@002
vertex_ai/code-bison-32k
vertex_ai/code-bison-32k@002
vertex_ai/code-gecko@latest
vertex_ai/code-gecko@001
vertex_ai/code-gecko@002
vertex_ai/codechat-bison@latest
vertex_ai/codechat-bison@001
vertex_ai/codechat-bison@002
vertex_ai/codechat-bison-32k
vertex_ai/codechat-bison-32k@002
vertex_ai/gemini-1.0-pro
vertex_ai/gemini-1.0-pro-001
vertex_ai/gemini-1.0-pro-002
vertex_ai/gemini-1.0-ultra
vertex_ai/gemini-1.0-ultra-001
vertex_ai/gemini-1.5-pro
vertex_ai/gemini-1.5-pro-001
vertex_ai/gemini-1.5-flash
vertex_ai/gemini-1.5-flash-001

Thank you

Motivation, pitch

I know that some models will be EOL soon, but we are testing multiple services to centralize the model access, and we have access to all models tested.

Twitter / LinkedIn details

No response

ishaan-jaff commented 1 week ago

working on this @04cfb1ed

can I see your config.yaml
what is the error you see for the models not working ?

ishaan-jaff commented 1 week ago

Okay I see we're missing this in our model cost map

[x] vertex_ai/text-bison@002
[x] vertex_ai/text-bison-32k
[x] vertex_ai/text-bison-32k@002
[x] vertex_ai/chat-bison-32k@002
[x] vertex_ai/code-bison@002
[x] vertex_ai/code-bison-32k
[x] vertex_ai/code-bison-32k@002
[x] vertex_ai/code-gecko@latest
[x] vertex_ai/codechat-bison@latest
[x] vertex_ai/codechat-bison@002
[x] vertex_ai/codechat-bison-32k@002
[x] vertex_ai/gemini-1.0-ultra
[x] vertex_ai/gemini-1.0-ultra-001

ishaan-jaff commented 1 week ago

done here @04cfb1ed

https://github.com/BerriAI/litellm/pull/4421

ishaan-jaff commented 1 week ago

@04cfb1ed any chance we can hop on a call ? I'd love to learn how how we can improve litellm for you.

Sharing a link to my cal for your convenience: https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat My linkedin if you prefer DMs: https://www.linkedin.com/in/reffajnaahsi/

BerriAI / litellm