search

openai / openai-dotnet

The official .NET library for the OpenAI API
https://www.nuget.org/packages/OpenAI/AbsoluteLatest
MIT License
1.03k stars 97 forks source link

BUG: MessageContent.FromImageUrl(Uri) incompatible with "data url" #135

Open crickman opened 1 month ago

crickman commented 1 month ago

Open uses a custom URL format for providing an image message where the base64 encoded image bytes are provided:

data:image/jpeg;base64,{base64_image}

The .Net Uri constructor has a length limitation.

joseharriaga commented 1 month ago

Thank you for reaching out, @crickman ! The documentation that you're linking to refers to the Chat Completions API, while the MessageContent class that you mention is part of the Assistants API instead. For Chat Completions, we have an example here of how to provide an image as BinaryData as part of the content of a UserChatMessage: 🔗 https://github.com/openai/openai-dotnet/blob/main/examples/Chat/Example05_ChatWithVision.cs

crickman commented 1 month ago

@joseharriaga - Correct, I am identifying an issue with the Assistant API The documentation I liked was soley for the purpose of identifying the "data-url" format.

image

The assistant code I am referencing is:

https://github.com/openai/openai-dotnet/blob/main/src/Custom/Assistants/MessageContent.cs#L27

Adding a factory method that accepts BinaryData would be superb!

sibaji-vayu commented 1 month ago

@joseharriaga I'm facing the same issue with OpenAI Assistant. I'm unable to post base64-encoded binary image data using the .NET SDK. Any update on this?