Question: GPT4-vision

[The-Michael-Chen]

Hey! How would I use gpt4-vision with this library? I'd like to pass in an image and text asking about the image and get the response.

[The-Michael-Chen]

I believe I figured it out! Was this in the documentation anywhere? I had trouble finding it. I had to dig through the commit history to find this. val request = chatCompletionRequest { model = ModelId("gpt-4-vision-preview") messages { user { content { text("What’s in this image?") image("https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg") } } } maxTokens = 300 }

[The-Michael-Chen]

Also, is there a way to pass in a local image instead of the url?

[BonnenuIt]

Same question. The authors provide some codes in openai-client/src/jvmTest/kotlin/com/aallam/openai/client/TestChatVisionJVM.kt. But I don't know how to make a ChatMessage list in this way.

[BonnenuIt]

        val reqList: ArrayList<ContentPart> = ArrayList<ContentPart>()
        reqList.add(TextPart("Hello! Describe the image for me."))
        reqList.add(ImagePart("data:image/jpeg;base64,$picInBase64"))
        val chatCompletionRequest = ChatCompletionRequest(
            model = ModelId("gpt-4-vision-preview"),
            messages = listOf(
                ChatMessage(
                    role = ChatRole.System,
                    content = "You are a helpful assistant!"
                ),
                ChatMessage(
                    role = ChatRole.User,
                    content = reqList
                )
            )
        )

↑ works.