[Feature]: Support adding vertex models via `/model/new`

[krrishdholakia]

The Feature

Allow adding new vertex models via the /model/new endpoint

Motivation, pitch

allow user to not need to add vertex models via the config

Twitter / LinkedIn details

No response

[krrishdholakia]

Vertex credentials will need to be passed in.

User can pass this in as a json.

We will then need to store this json in the db, and have a client initialized in the router (or dynamically initialized when call is made)

[krrishdholakia]

assuming the credentials are a service_account.json,

• need to accept a service_account.json as a completion() param
• create a credentials object from the json string
• init the vertex client with the credentials object

[krrishdholakia]

Putting sample code here:

import google.oauth2.service_account

json_obj = json.loads(service_account_json_str) 

credentials = google.oauth2.service_account.Credentials.from_service_account_info(json_obj)

vertexai.init(project="vertex_project", location="vertex_location", credentials=credentials)

### CODE FOR TESTING ### 
from vertexai.generative_models import GenerativeModel, Part

multimodal_model = GenerativeModel("gemini-1.0-pro-vision")
# Query the model
response = multimodal_model.generate_content(
    [
        # Add an example image
        Part.from_uri(
            "gs://generativeai-downloads/images/scones.jpg", mime_type="image/jpeg"
        ),
        # Add an example query
        "what is shown in this image?",
    ]
)
print(response)

[krrishdholakia]

curl --location 'http://0.0.0.0:4000/model/new' \
--header 'Authorization: Bearer sk-my-api-key' \ # 👈 ADMIN KEY
--header 'Content-Type: application/json' \
--data '{
  "model_name": "gemini",
  "litellm_params": {
     "model": "vertex_ai/gemini-pro",
     "vertex_project": "vertex-project",
     "vertex_location": "us-east-1",
     "vertex_credentials": json.dumps(service_account_obj.json) 
  }
}'

[krrishdholakia]

live in v1.35.6