Open ishaan-jaff opened 1 month ago
i believe this is already supported via model garden - https://docs.litellm.ai/docs/providers/vertex#model-garden
@krrishdholakia it is not supported, the Vertex API throws an error:
{"error":{"message":"VertexAIException - 400 Failed to deserialize the JSON body into the target type: instances[0]: missing field `inputs` at line 1 column 74
FYI this is my minimal reproducible setup:
import litellm
litellm.vertex_project = "0000" # project id
litellm.vertex_location = "us-central1"
response = litellm.completion(
model="vertex_ai/0000", # endpoint id
messages=[{"role": "user", "content": "write code for saying hi from LiteLLM"}],
)
print(response)
My guess is it's because it's a hugging face model that's deployed to vertex
The Feature
user is trying to use litellm for calling VertexAI / hugging face model
Motivation, pitch
-
Twitter / LinkedIn details
No response