Image URLs are only allowed for messages with role 'user', but this message with role 'assistant' contains an image URL.

[XStarlink]

Confirm this is a Node library issue and not an underlying OpenAI API issue

• [X] This is an issue with the Node library

Describe the bug

Hello,

I have a very simple process that I think used to work with GPT-4 turbo, or a previous version of this library, but that no longer works because of a rule in the nodejs lib.

Everything happens in the same chat:

1. I create a chat in my chatbot app
2. I make Dalle 3 generate an image
3. I ask a question to GPT-4mini, I get the error:

handleLLMError 400 Invalid 'messages[2]'. Image URLs are only allowed for messages with role 'user', but this message with role 'assistant' contains an image URL.

passing this message array:

[
  {
    "role": "user",
    "content": "Draw a cat"
  },
  {
    "role": "assistant",
    "content": [
      {
        "type": "image_url",
        "image_url": {
          "url": "[URL of generated image stored somewhere]"
        }
      }
    ]
  },
  {
    "role": "user",
    "content": "Describe this image"
  }
]

I don't think it's logical not to be able to describe the image generated by Dalle 3 in GPT-4o. What do you think?

To Reproduce

Everything happens in the same chat:

1. I create a chat in my chatbot app
2. I make Dalle 3 generate an image
3. I ask a question to GPT-4mini, I get the error:

handleLLMError 400 Invalid 'messages[2]'. Image URLs are only allowed for messages with role 'user', but this message with role 'assistant' contains an image URL.

Code snippets

No response

OS

macOS

Node version

node 21

Library version

openai 4.57.0

[RobertCraigie]

Thanks for reporting!

This sounds like an issue with the underlying OpenAI API and not the SDK, so I'm going to go ahead and close this issue.

Would you mind reposting at community.openai.com?