openai / openai-openapi

OpenAPI specification for the OpenAI API
https://platform.openai.com/docs/api-reference/introduction
MIT License
1.3k stars 378 forks source link

CreateFileRequest does not include "vision" purpose #260

Open phronmophobic opened 6 months ago

phronmophobic commented 6 months ago

Use "assistants" for Assistants and Message files, "vision" for Assistants image file inputs,

The purpose documentation says "vision" is a valid purpose, but the spec does not include it: https://github.com/openai/openai-openapi/blob/893ba52242dbd5387a97b96444ee1c742cfce9bd/openapi.yaml#L8603

dewmguy commented 6 months ago

https://platform.openai.com/docs/api-reference/files/create

documentation clearly states

"vision" for Assistants image file inputs

https://platform.openai.com/docs/api-reference/files/object#files/object-purpose

documentation also states vision is an appropriate use for the 'purpose' enum.

i'm under the assumption that my code (snippet below) should be working when using the vision purpose

const isImage = file.name.match(/\.(jpeg|jpg|webp|png)$/i);
const filePurpose = isImage ? 'vision' : 'assistants';
const fileData = await openai.files.create({ file: fileObject, purpose: filePurpose });

but i get this error:

Error: 400 Files with extensions [.png] are not supported for retrieval. See https://platform.openai.com/docs/assistants/tools/file-search/supported-files.

should i be using urls rather than trying to push the file buffer like i would for any other file upload purpose?

redvivi commented 6 days ago

Any update on this issue ?

furrykef commented 5 days ago

I was having the same problem. The correct way to add an image to a thread is to add a message with a content type of image_file: https://platform.openai.com/docs/assistants/deep-dive#creating-image-input-content