[Feature Request] Support Anthropic Model on Google Vertex

[ryanhex53]

🥰 Feature Description

Currently, I can't get it to work with Claude 3.5 on Google Cloud Vertex AI. Google Cloud offers a $300 no-cost trial credit for 90 days. Could anyone help with this?

🧐 Proposed Solution

Here are some information from google cloud documentation:

Note that the API for Claude on Vertex differs from the Anthropic API documentation in the following ways:

• model is not a valid parameter. The model is instead specified in the Google Cloud endpoint URL.
• anthropic_version is a required parameter and must be set to "vertex-2023-10-16"

request.json

{
  "anthropic_version": "vertex-2023-10-16",
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "image",
          "source": {
            "type": "base64",
            "media_type": "image/png",
            "data": "iVBORw0KGg..."
          }
        },
        {
          "type": "text",
          "text": "What is in this image?"
        }
      ]
    }
  ],
  "max_tokens": 256,
  "stream": true
}

PROJECT_ID=your-gcp-project-id
MODEL=claude-3-5-sonnet-v2@20241022

# Pick one region:
LOCATION=us-east5
# LOCATION=europe-west1

curl -X POST \
  -H "Authorization: Bearer $(gcloud auth print-access-token)" \
  -H "Content-Type: application/json; charset=utf-8" \
  -d @request.json \
"https://$LOCATION-aiplatform.googleapis.com/v1/projects/$PROJECT_ID/locations/$LOCATION/publishers/anthropic/models/$MODEL:streamRawPredict"

📝 Additional Information

No response

[ryanhex53]

I'm working on this now

[ryanhex53]

5794 work done, merge pending.