uezo / aiproxy

🦉AIProxy is a reverse proxy for ChatGPT API that provides monitoring, logging, and filtering requests and responses.
Apache License 2.0
38 stars 1 forks source link

Add experimental support for Claude on Amazon Bedrock #33

Closed uezo closed 3 months ago

uezo commented 3 months ago

To use Claude on Amazon Bedrock, use BedrockClaudeProxy instead of ClaudeProxy.

from aiproxy.bedrock_claude import BedrockClaudeProxy

bedrock_claude_proxy = BedrockClaudeProxy(
    aws_access_key_id="YOUR_AWS_ACCESS_KEY_ID",
    aws_secret_access_key="YOUR_AWS_SECRET_ACCESS_KEY",
    region_name="YOUR_REGION",
    access_logger_queue=worker.queue_client
)
bedrock_claude_proxy.add_route(app)

Client side. We test API with AnthropicBedrock.

# Make client with `base_url`
client = anthropic.AnthropicBedrock(
    aws_secret_key="dummy_aws_secret_access_key",
    aws_access_key="dummy_aws_access_key_id",
    aws_region="dummy_region_name",
    base_url="http://127.0.0.1:8000/bedrock-claude"
)

resp = client.messages.create(
    model="anthropic.claude-3-haiku-20240307-v1:0",
    messages=[{"role": "user", "content": [{"type": "text", "text": "こんにちは!"}]}],
    max_tokens=512,
    stream=True
)

for r in resp:
    print(r)