Response type should support multiple messages

nileshtrivedi commented 1 month ago

I am using a custom chat endpoint where the request is handled by a group of agents, and therefore, the response comes as a list of messages. It looks like DeepChat is only prepared to handle a single message in the Response: https://deepchat.dev/docs/connect/#Response

It would be great if this assumption can be relaxed.

mchill commented 1 month ago

I would also like this, but in case you haven't seen it, there's a workaround using the websocket handler.

nileshtrivedi commented 1 month ago

@mchill Interesting but this is going to be tricky for me as my endpoint is served by ReactPy in FastAPI so I actually have no easy way to create Javascript objects. This is my UI endpoint in pure Python:

@component
def Agent():
    return DeepChat({
        "request": {"url": "http://localhost:8000/chat"}
    })

OvidijusParsiunas commented 1 month ago

Hey folks, apologies for the late replies as I am currently very occupied with my current job.

I wanted to offer an alternative, you could potentially intercept the incoming responses using the responseInterceptor, parse the content, and use a method called _addMessage multiple times instead. This method is not part of our official documentation and will be added to the main API on our next release, however it is still available in the current release. The way it works is it accepts a Response object as an argument and adds a message to the chat. E.g. deepChatRef._addMessage({text: 'hello'});

Let me know if this helps. Thanks!

nileshtrivedi commented 1 month ago

@OvidijusParsiunas There is a gotcha with responseInterceptor. It is still expected to return a valid Response object which ends up repeating one of the messages. Returning null from responseInterceptor is treated as an Error.

<html>
    <head>
        <script src="https://unpkg.com/deep-chat@1.4.11/dist/deepChat.bundle.js" type="module" crossorigin></script>
    </head>
    <body>
        <deep-chat id="chat" demo="true" request='{"url": "/chat"}' stream='{"simulation": 6}'></deep-chat>
        <script>
            window.onload = function() {
                let chatElementRef = document.getElementById('chat');
                chatElementRef.responseInterceptor = (response) => {
                    console.log(response); // printed above
                    response.forEach(m => chatElementRef._addMessage(m))
                    return response[0];
                };
            };
        </script>
    </body>
</html>

nileshtrivedi commented 1 month ago

I am now calling _addMessage for all messages except the last one and returning the last one from the responseInterceptor. This works:

[
  {
    "text": "Cathy, tell me a joke.",
    "role": "joe"
  },
  {
    "text": "Sure, here's one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!",
    "role": "cathy"
  }
]

            window.onload = function() {
                let chatElementRef = document.getElementById('chat');
                chatElementRef.responseInterceptor = (response) => {
                    console.log(JSON.stringify(response, null, 2)); 
                    response.slice(0, response.length - 1).forEach(m => chatElementRef._addMessage(m))
                    return response[response.length - 1];
                };
            };

Now there is just a minor glitch that the role from the last message is not respected.

OvidijusParsiunas commented 1 month ago

Hi @nileshtrivedi.

It is strange that the returned role is not respected.

Could you perhaps make the last message returned by the interceptor define an explicit role?

E.g. return {role: 'your-role-name', text: response[response.length - 1].text}

nileshtrivedi commented 1 month ago

@OvidijusParsiunas That does not fix it.

Here is the whole app in a single file. You can run this as OPENAI_API_KEY=api_key fastapi dev app.py after pip install autogen fastapi fastapi-cli:

import os
import json

import tempfile
from typing import Annotated, Literal
from pathlib import Path
from autogen import ConversableAgent, register_function
from autogen.coding import LocalCommandLineCodeExecutor

from fastapi import FastAPI, Body, Request, WebSocket
from fastapi.responses import HTMLResponse
from fastapi.staticfiles import StaticFiles

# from pydantic import BaseModel, List
# from prefect import flow

openai_api_key = os.environ.get("OPENAI_API_KEY")

cathy = ConversableAgent(
    "cathy",
    system_message="Your name is Cathy and you are a part of a duo of comedians.",
    llm_config={"config_list": [{"model": "gpt-4", "temperature": 0.9, "api_key": openai_api_key}]},
    human_input_mode="NEVER",  # Never ask for human input.
)

joe = ConversableAgent(
    "joe",
    system_message="Your name is Joe and you are a part of a duo of comedians.",
    llm_config={"config_list": [{"model": "gpt-4", "temperature": 0.7, "api_key": openai_api_key}]},
    human_input_mode="NEVER",  # Never ask for human input.
)

app = FastAPI()

@app.get("/", response_class=HTMLResponse)
async def root():
    homepage_html = """
<html>
    <head>
        <script src="https://unpkg.com/deep-chat@1.4.11/dist/deepChat.bundle.js" type="module" crossorigin></script>
    </head>
    <body>
        <deep-chat id="chat" demo="true" request='{"url": "/chat"}' stream='{"simulation": 6}' names="true"></deep-chat>
        <script>
            window.onload = function() {
                let chatElementRef = document.getElementById('chat');
                chatElementRef.responseInterceptor = (response) => {
                    console.log(JSON.stringify(response, null, 2)); 
                    response.slice(0, response.length - 1).forEach(m => chatElementRef._addMessage(m))
                    console.log(response[response.length - 1])
                    return {role: "cathy", text: response[response.length - 1].text};
                };
            };
        </script>
    </body>
</html>
    """
    return homepage_html

@app.post("/chat")
async def chat(request: Request):
    data = await request.json()
    user_msg = data["messages"][0]["text"]

    result = joe.initiate_chat(cathy, message="Cathy, tell me a joke.", max_turns=1)
    return [{"text": m["content"], "role": ("joe" if m["role"] == "assistant" else "cathy")} for m in result.chat_history]

buzhou9 commented 1 month ago

You can try this, it will correctly identify the identity. This is the document about this parameter. https://deepchat.dev/docs/messages/#names In addition, you need to delete "stream={" simulation ": 6} '" because it is used to simulate AI streaming response, it is only debugging code, and does not consider this special use case. The role parameter is ignored and fixed to the default' ai '

<deep-chat id="chat" demo="true"  names='{"user":{"text":"cathy"}}'></deep-chat>
<script>
  window.onload = function() {
    let chatElementRef = document.getElementById('chat');
    chatElementRef.responseInterceptor = (response) => {
      response = [
        {
          "text": "Cathy, tell me a joke.",
          "role": "joe"
        },
        {
          "text": "Sure, here's one for you:\n\nWhy don't scientists trust atoms?\n\nBecause they make up everything!",
          "role": "user"
        }
      ]
      console.log(JSON.stringify(response, null, 2));
      response.slice(0, response.length - 1).forEach(m => chatElementRef._addMessage(m))
      console.log(response[response.length - 1])
      return response[response.length - 1];
    };
  };
</script>

nileshtrivedi commented 1 month ago

@buzhou9 Yes, deleting "stream={" simulation ": 6} " indeed fixes the names. Thanks again to both you and @OvidijusParsiunas ! 👍

OvidijusParsiunas / deep-chat

Response type should support multiple messages #191