[Bug]: RetrieveUserProxyAgent doesn't seem to work with pyautogen==0.2.20

maurominella commented 5 months ago

Describe the bug

I'm trying to test RAG with autogen, following the documentation at https://microsoft.github.io/autogen/blog/2023/10/18/RetrieveChat/.

I installed pyautogen==0.2.20, "pyautogen[retrievechat]", pypdf and "unstructured[all-docs]".

The problem is that when I run ragproxyagent.initiate_chat(assistant, problem="What is autogen?")

, it shows a textbox to gather a string:

Shouldn't it start running the agent to extract the answer from the page provided, as shown in the documentation ? I don't understand what it expects from me.

If I simply hit enter, it replies asking me to specify a real question. If I repeat the string that I passed as parameter, it provides a generic answer that does not take into account the link to the .md file that I passed when I created the "RetrieveUserProxyAgent" object:

Just for testing I tried to explicitly call ragproxyagent.retrieve_docs(problem="What is autogen?", n_results=3,search_string="")

, which instead started opening the link and doing some tasks (chunking and embedding, I guess)

In other words, it seems that the "problem" parameter is not considered at all, or that the RAG tasks simply do not start

Steps to reproduce

Follow the steps indicated by the documentation, using the (latest, as of today) packages that I installed:

Step 1:

import autogen
from autogen.agentchat.contrib.retrieve_assistant_agent import RetrieveAssistantAgent
from autogen.agentchat.contrib.retrieve_user_proxy_agent import RetrieveUserProxyAgent

Step 2:

assistant = RetrieveAssistantAgent(
    name="assistant",
    system_message="You are a helpful assistant.",
    llm_config=llm_config,
)

ragproxyagent = RetrieveUserProxyAgent(
    name="ragproxyagent",
    retrieve_config={
        "task": "qa",
        "docs_path": "https://raw.githubusercontent.com/microsoft/autogen/main/README.md",
    },
)

Step 3:

assistant.reset()
ragproxyagent.initiate_chat(assistant, problem="What is autogen?")

Model Used

Azure OpneAI service with deployment gpt4 turbo version 0125:

{'cache_seed': 41,
 'config_list': [{'model': 'gpt4-0125preview-128k',
   'api_key': '**************',
   'base_url': 'https://**********.openai.azure.com/',
   'api_type': 'azure',
   'api_version': '2024-02-15-preview'}],
 'temperature': 0}