deepset-ai / haystack-cookbook

👩🏻‍🍳 A collection of example notebooks
383 stars 71 forks source link

Cookbook showing RAG with references to documents in answers #73

Open julian-risch opened 8 months ago

julian-risch commented 8 months ago

Haystack 1.x had this prompt template https://prompthub.deepset.ai/?prompt=deepset%2Fquestion-answering-with-references instructing the LLM to provide references "[1]" to documents in the generated answer. It would be great to have a cookbook showing how to do that with Haystack 2.x.

The existing https://github.com/deepset-ai/haystack-cookbook/blob/main/notebooks/prompt_customization_for_Anthropic.ipynb is related but uses quotes from the documents. What I am looking for is a free text generated answer with references to documents not quotes.

Prompt could be the following:

from haystack.components.builders import PromptBuilder

template = """
Create a concise and informative answer (no more than 50 words) for a given question based solely on the given documents. 
You must only use information from the given documents. Use an unbiased and journalistic tone. Do not repeat text. 
Cite the documents using Document[number] notation. 
If multiple documents contain the answer, cite those documents like ‘as stated in Document[number], Document[number], etc.’. 
If the documents do not contain the answer to the question, say that ‘answering is not possible given the available information.’

Given the following information, answer the question.

{% for document in documents %}
    Document[{{loop.index}}]: {{ document.content }} \n
{% endfor %}

Question: {{question}}
Answer:
"""

prompt_builder = PromptBuilder(template=template)
julian-risch commented 8 months ago

I prepared a notebook here: https://colab.research.google.com/drive/1-WZcQlpB_h6C82VHeeSFcaWqRl15aDC-?usp=sharing