Open leventmolla opened 9 months ago
Correct, this is done with the ChatCompletions endpoint. The documentation indicates that the User role can send 3 types of Message Content:
{
"type": "text",
"text": text
}
{
"type": "image_url",
"image_url": {
"url": url or base64 encoded data,
"detail": "auto", "low" or, "high"
}
}
https://github.com/MacPaw/OpenAI/pull/169 adds the ChatCompletionContentPartImageParam type (as it's called in the Python API). I hadn't included base64 support, but can do so now.
looking at the Python code, OpenAI counts on you to encode image data to base64 string and feed that to the image_url.url parameter; looks like i had coded it that way because I was following the Python API precisely.
class ImageURL(TypedDict, total=False):
url: Required[str]
"""Either a URL of the image or the base64 encoded image data."""
detail: Literal["auto", "low", "high"]
"""Specifies the detail level of the image.
Learn more in the
[Vision guide](https://platform.openai.com/docs/guides/vision/low-or-high-fidelity-image-understanding).
"""
class ChatCompletionContentPartImageParam(TypedDict, total=False):
image_url: Required[ImageURL]
type: Required[Literal["image_url"]]
"""The type of the content part."""
see src/openai/types/chat/chat_completion_content_part_image_param.py
Discussed in https://github.com/MacPaw/OpenAI/discussions/147