oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.87k stars 5.34k forks source link

fix: openai extension #6528

Closed hronoas closed 1 week ago

hronoas commented 2 weeks ago

Pull Request Title: Fix: Handle Multiple Content Items in Messages

Description:

Currently, when messages contain multiple content items, only the last item in the list is used.

Example Issue:

Given the following JSON input:

{
  "messages": [
    {
      "role": "user",
      "content": [
        {
          "type": "text",
          "text": "first item"
        },
        {
          "type": "text",
          "text": "second item"
        },
        {
          "type": "text",
          "text": "last item"
        }
      ]
    }
  ]
}

The system currently utilizes only "last item" and ignores the other entries.

Proposed Fix:

This PR addresses the issue by ensuring all content items within a message are processed appropriately.

Checklist:

hronoas commented 2 weeks ago

For example, the Cline extension in VSCode does not work correctly due to this issue.

Ph0rk0z commented 2 weeks ago

Also some food for thought related to this: https://old.reddit.com/r/SillyTavernAI/comments/1govgfw/dev_st_produces_incorrect_chat_completion/

There may also be multiple SYSTEM messages depending on what front end/app is communicating with the API in chat completion. This may break the template for the model as well.

Chat completion sure is "fun".

oobabooga commented 1 week ago

Thanks for the fix