Anthropic Vertex Support

[VVoruganti]

Description

Support for Claude 3 model access via the GCP Vertex service. Similar implementation to the AnthropicBedrock provider.

[brenkao]

I will look into adding this!

[VVoruganti]

Testing out this implementation

def vertex_client_wrapper(
    project_id: Optional[str] = None,
    region: Optional[str] = None,
) -> Callable[
    [Union[Anthropic, AsyncAnthropic]], Union[AnthropicVertex, AsyncAnthropicVertex]
]:
    """Returns a client wrapper for using Anthropic models on GCP Vertex."""

    def inner_wrapper(client: Union[Anthropic, AsyncAnthropic]):
        """Returns matching `AnthropicVertex` or `AsyncAnthropicVertex` client."""
        kwargs = {"project_id": project_id, "region": region}
        if isinstance(client, Anthropic):
            client = AnthropicVertex(**kwargs)  # type: ignore
        elif isinstance(client, AsyncAnthropic):
            client = AsyncAnthropicVertex(**kwargs)  # type: ignore
        return client

    return inner_wrapper

[brenkao]

I've confirmed the above code works w/ model claude-3-haiku@20240307.

For reference the code I used to test this was

Call

class BookRecommender(AnthropicCall):
    prompt_template = "Please recommend a fantasy book."

    call_params = AnthropicCallParams(model="claude-3-haiku@20240307")
    configuration = BaseConfig(
        client_wrappers=[
            vertex_client_wrapper(
                project_id="MY_PROJECT_ID",
                region="us-central1",
            )
        ]
    )
recommender = BookRecommender()
response = recommender.call()
print(response.content)
# > Here are a few recommendations for fantasy books:

# 1. "The Name of the Wind" by Patrick Rothfuss - This is the first book in the Kingkiller Chronicle series, which is a critically acclaimed and immersive fantasy story about a legendary wizard-like figure telling the story of his life.
...

I also tested using AnthropicExtractor as well.

Would you like to submit the PR instead? @VVoruganti

[VVoruganti]

I've confirmed the above code works w/ model claude-3-haiku@20240307.

For reference the code I used to test this was

Call

class BookRecommender(AnthropicCall):
    prompt_template = "Please recommend a fantasy book."

    call_params = AnthropicCallParams(model="claude-3-haiku@20240307")
    configuration = BaseConfig(
        client_wrappers=[
            vertex_client_wrapper(
                project_id="MY_PROJECT_ID",
                region="us-central1",
            )
        ]
    )
recommender = BookRecommender()
response = recommender.call()
print(response.content)
# > Here are a few recommendations for fantasy books:

# 1. "The Name of the Wind" by Patrick Rothfuss - This is the first book in the Kingkiller Chronicle series, which is a critically acclaimed and immersive fantasy story about a legendary wizard-like figure telling the story of his life.
...

I also tested using AnthropicExtractor as well.

Would you like to submit the PR instead? @VVoruganti

Yes! I can submit a PR with a test in the next few hours