rawwerks / magentic

Seamlessly integrate LLMs as Python functions
https://magentic.dev/
MIT License
0 stars 0 forks source link

Implement `message_to_anthropic_message` for `UserImageMessage` #4

Open mentatbot[bot] opened 2 weeks ago

mentatbot[bot] commented 2 weeks ago

This commit adds support for converting UserImageMessage to the Anthropic message format. The implementation mirrors the existing message_to_openai_message function, ensuring compatibility with image messages for Anthropic models.

Changes include:

This enhancement allows the use of image messages with Anthropic models, similar to the existing support for OpenAI models.

Closes #3

rawwerks commented 2 weeks ago

@mentatbot - you didn't implement the API call correctly in "message_to_anthropic_message.register(UserImageMessage)"

here are two examples

import anthropic

client = anthropic.Anthropic()
message = client.messages.create(
    model="claude-3-5-sonnet-20240620",
    max_tokens=1024,
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": image1_media_type,
                        "data": image1_data,
                    },
                },
                {
                    "type": "text",
                    "text": "Describe this image."
                }
            ],
        }
    ],
)
print(message)
message = client.messages.create(
    model="claude-3-5-sonnet-20240620",
    max_tokens=1024,
    system="Respond only in Spanish.",
    messages=[
        {
            "role": "user",
            "content": [
                {
                    "type": "text",
                    "text": "Image 1:"
                },
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": image1_media_type,
                        "data": image1_data,
                    },
                },
                {
                    "type": "text",
                    "text": "Image 2:"
                },
                {
                    "type": "image",
                    "source": {
                        "type": "base64",
                        "media_type": image2_media_type,
                        "data": image2_data,
                    },
                },
                {
                    "type": "text",
                    "text": "How are these images different?"
                }
            ],
        }
    ],
)