MacPaw / OpenAI

Swift community driven package for OpenAI public API
MIT License
2.1k stars 351 forks source link

Support for gpt-4-vision-preview #119

Open jj0hns0n opened 1 year ago

jj0hns0n commented 1 year ago

Is there intended to be support for the new gpt-4-vision-preview model soon?

ingvarus-bc commented 1 year ago

Hi, @jj0hns0n, thanks for reaching out! ✨
The gpt-4-vision-preview will be added with PR-115, though Swift native support may follow in later releases, as there are a bunch of other more critical features to be covered first.

In the Documentation there are examples of how it had been implemented using Python but no direct API Reference. If you'd like to have this feature available sooner feel free to open Pull Request and contribute to the project, we always appreciate that! ☺️

thyselius commented 11 months ago

Does anyone have an example of uploading an image to the API together with a prompt for that image?

I'm trying to implement uploading an image to the API. However I get the following error when I'm trying this approach:

        ```
        var content = """
                    [
                        {
                          "type": "text",
                          "text": "\(text)"
                        },
                        {
                          "type": "image_url",
                          "image_url": {
                            "url": f"data:image/png;base64,\(base64!)"
                          }
                        }
                    ]
        """

        query = CompletionsQuery(model: .gpt4_vision_preview, prompt: content, temperature: 1, maxTokens: 4000, topP: 1, frequencyPenalty: 0, presencePenalty: 0, stop: ["\\n"])


            ERROR:
    `2 validation errors for Request\nbody -> prompt\n  value is not a valid list (type=type_error.list)\nbody -> prompt\n  value is not a valid list (type=type_error.list)", type: "invalid_request_error"`