[Bug]: RAG pipeline not updating contexts after 1st query [RAG]

[swaroop11]

Describe the bug

I have created a .py file with below code which uses RetrieveUserProxyAgent and RetrieveUserProxyAgent agents. I have kept all the settings to default only and initiated the chat with the ragproxyagent. For the first question it correctly pulls context and provide answer, but from next query onwards it just say "UPDATE CONTEXT" and generates no response.

Ideally, it should pull new context for every next query user is asking and generate response based on that, but not doing that.

Steps to reproduce

import autogen
from autogen.agentchat.assistant_agent import AssistantAgent
from autogen.agentchat.contrib.retrieve_assistant_agent import RetrieveAssistantAgent
from autogen.agentchat.contrib.retrieve_user_proxy_agent import RetrieveUserProxyAgent
from langchain.text_splitter import RecursiveCharacterTextSplitter

config_list = autogen.config_list_from_json("OAI_CONFIG_LIST",
                                            filter_dict={"model": ["gpt-4"],},)

gpt4_api_key = config_list[0]["api_key"]
os.environ['OPENAI_API_KEY'] = gpt4_api_key

text_splitter = RecursiveCharacterTextSplitter(chunk_size=00, chunk_overlap=0, separators=["\n"], keep_separator=False)

assistant = AssistantAgent(
    name="assistant",
    llm_config={
        "timeout": 600,
        "config_list": config_list,
    },
)

ragproxyagent = RetrieveUserProxyAgent(
    name="ragproxyagent",
    retrieve_config={
        "task": "qa",
        "custom_text_split_function": text_splitter.split_text,
        "docs_path": "context_file.txt",
    },
    code_execution_config=False,
)

assistant.reset()
ragproxyagent.initiate_chat(assistant, message=ragproxyagent.message_generator, problem="i want to stop getting notification what to do?",
                            `n_results=2)

### Model Used

gpt-4 

### Expected Behavior

It should have called RAG with new context for each input query, but that is not happening.

### Screenshots and logs

These are the logs, I removed sensitive info.
![image](https://github.com/microsoft/autogen/assets/37260686/2f55a4e4-70a3-4ff3-a303-6c8fd47e2b15)

### Additional Information

_No response_

[swaroop11]

[thinkall]

Hi @swaroop11 , if you'd like to ask for a series of questions, a recommended design would be:

while True:
    problem = input("Please enter your question: ")
    if problem == "":
        break
    assistant.reset()
    ragproxyagent.initiate_chat(assistant, message=ragproxyagent.message_generator, problem=problem)

[swaroop11]

Hi @thinkall, I checked your suggestion but it giving the same issue. If human_input_mode="ALWAYS" then RAG is performed only once and agent tries to answer all the future questions using that context only. Ideally it should pull new context for each input query. So the issue is not solved

ragproxyagent = RetrieveUserProxyAgent( name="ragproxyagent", human_input_mode="ALWAYS", max_consecutive_auto_reply=2, retrieve_config={ "task": "qa", "custom_text_split_function": text_splitter.split_text, "docs_path": "context_file.txt", }, code_execution_config=False, )

[Josephrp]

If human_input_mode="ALWAYS" then RAG is performed only once and agent tries to answer all the future questions using that context only. Ideally it should pull new context for each input query. So the issue is not solved

hey there @swaroop11 autogen is a bit wierd because it didnt start with rag, and i do think it "inherited" some idiosyncracies there, for example the importance of a good system prompt.

check out this demo on huggingface to see a good example for that. hope this helps : https://huggingface.co/spaces/thinkall/autogen-demos

[thinkall]

Hi @thinkall, I checked your suggestion but it giving the same issue. If human_input_mode="ALWAYS" then RAG is performed only once and agent tries to answer all the future questions using that context only. Ideally it should pull new context for each input query. So the issue is not solved

ragproxyagent = RetrieveUserProxyAgent( name="ragproxyagent", human_input_mode="ALWAYS", max_consecutive_auto_reply=2, retrieve_config={ "task": "qa", "custom_text_split_function": text_splitter.split_text, "docs_path": "context_file.txt", }, code_execution_config=False, )

Hi @swaroop11 , When you set human_input_mode="ALWAYS", for each question, when assistant replied, you will be prompted to input your reply (human_input); after the interaction finished, you'll be prompted to input the next question and start a new round of chat.