how to use vertex gemini with Proxy?

[krrishdholakia]

Hey @tgyy1995, do you have vertex ai credentials stored in your server?

[krrishdholakia]

you'll need to store your credentials like this

[tgyy1995]

I can use Gemini in this way:

from litellm import completion
import os
# auth: run 'gcloud auth application-default'
os.environ["VERTEX_PROJECT"] = "hardy-device-386718"
os.environ["VERTEX_LOCATION"] = "us-central1"
response = completion(
  model="chat-bison", 
  messages=[{ "content": "Hello, how are you?","role": "user"}]
)

But I want to use Gemini in this way and i dont kown how. For example, litellm --model huggingface/bigcode/starcoder. How should I set up VERTEX_PROJECT and VERTEX_LOCATION?

[krrishdholakia]

Do this in your terminal before running the liteLLM command.

export VERTEX_PROJECT="hardy-project"
export VERTEX_LOCATION="us-west"

[tgyy1995]

Thank you.

[ishaan-jaff]

@tgyy1995 added docs on how to do this too: https://docs.litellm.ai/docs/providers/vertex#gemini-pro

[ishaan-jaff]

@tgyy1995 what are you using the LiteLLM proxy for ?

[tgyy1995]

Thank you for your work. I can now use Gemini. I use Litellm for autogen.