The retriever tool only returns 1 Document back even top_k=3 is specified

[faaany]

Describe the bug When running my agent application using 2 tools, I found that the retriever tool always returns 1 Document back, even when I specify my retriever to return the top 3 Documents back. As you can see in my code shown below, I set the top_k in the BM25Retriever to 3:

from haystack.agents import Tool
from haystack.nodes import PromptNode
from haystack.agents.base import Agent, ToolsManager
from haystack.document_stores import InMemoryDocumentStore
from haystack.nodes import PromptNode, PromptTemplate, AnswerParser, BM25Retriever

from prompt_templates import agent_prompt_with_memory, qa_prompt_tempalte 

openai_api_key = 'xxxxx'

def launch_document_store():

    dataset = load_dataset("bilgeyucel/seven-wonders", split="train")
    document_store = InMemoryDocumentStore(use_bm25=True)
    document_store.write_documents(dataset)

    return document_store 

def resolver_function(query, agent, agent_step):
    return {
            "query": query,
            "tool_names_with_descriptions": agent.tm.get_tool_names_with_descriptions(),
            "transcript": agent_step.transcript,
            "memory": agent.memory.load(),
        }

if __name__ == "__main__":
        document_store = launch_document_store()

        retriever = BM25Retriever(document_store=document_store, top_k=3)

        prompt_node = PromptNode(model_name_or_path="text-davinci-003", api_key=openai_api_key, default_prompt_template=qa_prompt_template)

        retriever_tool = Tool(
                    name="paragraph_retriever",
                    pipeline_or_node=retriever,
                    description='useful when you need to find relevant paragraphs that might contain answers to a question about the seven wonders of the world',
                    output_variable="documents",
                )

        qa_tool = Tool(
                    name="question_answering_bot",
                    pipeline_or_node=prompt_node,
                    description="useful when you need to answer a question from a given documents",
                    output_variable="answers",
                )

        tools_manager = ToolsManager([retriever_tool, qa_tool])

        memory_prompt_node = PromptNode(
                "philschmid/bart-large-cnn-samsum", max_length=256, model_kwargs={"task_name": "text2text-generation"}
            )
        memory = ConversationSummaryMemory(memory_prompt_node, prompt_template="{chat_transcript}")

        agent_prompt_node = PromptNode(
            "gpt-3.5-turbo",
            api_key=openai_api_key,
            max_length=256,
            stop_words=["Observation:"],
            model_kwargs={"temperature": 0.5},
        )

        conversational_agent = Agent(
                    agent_prompt_node,
                    prompt_template=agent_prompt_with_memory,
                    prompt_parameters_resolver=resolver_function,
                    memory=memory,
                    tools_manager=tools_manager,
                )

        conversational_agent.run("What did the Rhodes Statue look like?")

This is the output I got:

The white text shows all 3 Documents the retriever returns, and the orange text is what the retriever tool returns.

Expected behavior I would expect that the retriever tool returns all 3 documents back because sometimes the correct answer is within the top2 document, not the top1 document. The drawback of this approach is that the agent LLM model will give a very long prompt text, which might lead to a slower response from the agent.

To Reproduce You can reproduce the behavior using my script above, but you need to insert your openai_api_key and put prompt templates in a seperate Python file. Examples of PromptTemplate for agent can be found in this tutorial.

FAQ Check

• [x] Have you had a look at our new FAQ page?

System:

• OS: Linux
• GPU/CPU: CPU
• Haystack version (commit or version number): the latest
• DocumentStore: InMemoryDocumentStore
• Reader:
• Retriever: BM25Retriever

[faaany]

use a callable retriever instead. See this PR.