Issue: How to build a prompt that include SystemMessage, vector store context and final question

nbrosson commented 1 year ago

Issue you'd like to raise.

Hello guys !

I would like to build a Prompt which include a System message, Context data from a vector_store and the final question. What is the best way to do that?

Right now, I'm doing this:

def get_prompt(system_prompt):
    prompt = SystemMessage(content=system_prompt)
    new_prompt = (
    prompt
    + "--- \n\n" + "le contexte:" + "\n"
    + "{context}"
    + '\n\n --- \n\n ' + "la question: \n\n"
    + "{question}"
    )
    return PromptTemplate(
    template=new_prompt, input_variables=["context", "question"]
    )

But get this error: KeyError: 'template'

Is there a way to do what I want? I'm searching for a solution where I don't need to add the context myself, because it should already be managed by the vector store retriever

Suggestion:

No response

dosubot[bot] commented 1 year ago

Answer generated by a 🤖

Answer

Bonjour,

Pour construire une invite dans LangChain qui inclut un message système, des données de contexte d'un vector_store et une question finale, vous pouvez utiliser la classe PromptTemplate fournie par le framework LangChain.

La classe PromptTemplate prend deux arguments : 'template' et 'input_variables'. L'argument 'template' est une chaîne de caractères qui définit la structure de l'invite, et 'input_variables' est une liste de noms de variables qui seront remplacés dans la chaîne de caractères du template.

Voici un exemple de comment vous pouvez construire une invite :

from langchain.prompts import PromptTemplate

# Définir la chaîne de caractères du template
template = """
Message système : {system_message}
Données de contexte : {context_data}
Question finale : {final_question}
"""

# Définir les variables d'entrée
input_variables = ["system_message", "context_data", "final_question"]

# Créer le PromptTemplate
prompt = PromptTemplate(template=template, input_variables=input_variables)

Dans cet exemple, 'system_message', 'context_data' et 'final_question' sont des placeholders dans la chaîne de caractères du template qui seront remplacés par des valeurs réelles lorsque vous utiliserez le PromptTemplate pour générer une invite.

Pour éviter une KeyError pour 'template', assurez-vous que l'argument 'template' est une chaîne de caractères et que tous les placeholders dans la chaîne de caractères du template sont inclus dans la liste 'input_variables'.

Il est également important de noter que la classe PromptTemplate prend en charge l'initialisation partielle avec l'argument 'partial_variables'. Cela vous permet de fournir certaines des variables d'entrée au moment de l'initialisation, et le reste plus tard lorsque vous appelez la méthode 'format'. Par exemple :

# Initialiser le PromptTemplate avec une variable partielle
prompt = PromptTemplate(
    input_variables=["context_data", "final_question"], 
    template=template, 
    partial_variables={"system_message": "Démarrage du système"}
)

# Plus tard, fournir le reste des variables d'entrée
prompt_text = prompt.format(context_data="Données du vecteur", final_question="Quel est votre nom ?")

Dans cet exemple, la variable 'system_message' est fournie au moment de l'initialisation, et les variables 'context_data' et 'final_question' sont fournies plus tard.

De plus, la classe PromptTemplate fournit également des méthodes pour créer une invite à partir d'exemples, à partir d'un fichier, ou à partir d'un template. La méthode 'from_template' est particulièrement utile lorsque vous voulez charger un template d'invite à partir d'une chaîne de caractères, et elle prend en charge les formats f-string et jinja2. Voici un exemple de comment l'utiliser :

# Charger un template d'invite à partir d'une chaîne de caractères
prompt = PromptTemplate.from_template(
    template="Message système : {system_message}\nDonnées de contexte : {context_data}\nQuestion finale : {final_question}", 
    partial_variables={"system_message": "Démarrage du système"}
)

# Plus tard, fournir le reste des variables d'entrée
prompt_text = prompt.format(context_data="Données du vecteur", final_question="Quel est votre nom ?")

Dans cet exemple, la variable 'system_message' est fournie au moment de l'initialisation, et les variables 'context_data' et 'final_question' sont fournies plus tard. La méthode 'from_template' extrait automatiquement les noms de variables de la chaîne de caractères du template, donc vous n'avez pas besoin de fournir l'argument 'input_variables'.

J'espère que cela vous aide à résoudre votre problème. Si vous avez d'autres questions, n'hésitez pas à les poser.

Cordialement, Dosu

Sources

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

nbrosson commented 1 year ago

I'm not sure this can help because this is not a real system message?

dosubot[bot] commented 1 year ago

Answer generated by a 🤖

Answer

Hello,

Based on the context you've provided, it seems you're trying to include a real system message in your prompt without manually adding the context. I found a similar issue in the LangChain repository that might help you solve your problem.

To include a system message with each call to 'qa.run(prompt)', you can use an agent to instruct it about its basic purpose. The best type of agent in this case will be CHAT_CONVERSATIONAL_REACT_DESCRIPTION. Here is the code:

from langchain.chains.conversation.memory import ConversationBufferWindowMemory
from langchain.chains import RetrievalQA
from langchain.chat_models import ChatOpenAI
from langchain.schema import SystemMessage
from langchain.agents.types import AgentType
from langchain.agents import initialize_agent
from langchain.tools import Tool
from langchain.prompts import MessagesPlaceholder

llm = ChatOpenAI(model_name="gpt-3.5-turbo", temperature=0.0)

conversational_memory = ConversationBufferWindowMemory(
    memory_key="chat_history", return_messages=True
)

qa = RetrievalQA.from_chain_type(
    llm=llm,
    chain_type="stuff",
    retriever=docsearch.as_retriever(),
)

system_message = """
    You are a Virtual Vet. "
    "You should help clients with their concerns about their pets and provide helpful solutions."
    "You can ask questions to help you understand and diagnose the problem."
    "You should only talk within the context of problem."
    "If you are unsure of how to help, you can suggest the client to go to the nearest clink of their place."
    "You should talk on German, unless the client talks in English."
    """
tools = [
    Tool(
        name="qa-vet",
        func=qa.run,
        description="Useful when you need to answer vet questions",
    )
]
executor = initialize_agent(
    agent = AgentType.CHAT_CONVERSATIONAL_REACT_DESCRIPTION,
    tools=tools,
    llm=llm,
    memory=conversational_memory,
    agent_kwargs={"system_message": system_message},
    verbose=True,
)

For the issue OutputParserException: Could not parse LLM output: {LLM reply gets printed here}, it can be solved by catching this exception using a try and except block as suggested in this link: https://github.com/langchain-ai/langchain/issues/3750.

To modify your LangChain code to include a real system message in the prompt without encountering a KeyError: 'template', you can adjust the DEFAULT_REFINE_PROMPT_TMPL or DEFAULT_TEXT_QA_PROMPT_TMPL in your refine_prompts.py file. You can add a new variable for the system message in the template and include it in the input_variables list when creating the PromptTemplate instance.

Here is an example of how you can modify your code:

from langchain.prompts import PromptTemplate

# Add a new variable for the system message in the template
DEFAULT_REFINE_PROMPT_TMPL = (
    "System Message: {system_message}\n"
    "The original question is as follows: {question}\n"
    "We have provided an existing answer, including sources: {existing_answer}\n"
    "We have the opportunity to refine the existing answer"
    "(only if needed) with some more context below.\n"
    "------------\n"
    "{context_str}\n"
    "------------\n"
    "Given the new context, refine the original answer to better "
    "answer the question. "
    "If you do update it, please update the sources as well. "
    "If the context isn't useful, return the original answer."
)

# Include the new variable in the input_variables list
DEFAULT_REFINE_PROMPT = PromptTemplate(
    input_variables=["system_message", "question", "existing_answer", "context_str"],
    template=DEFAULT_REFINE_PROMPT_TMPL,
)

In this modified code, the system_message variable is added to the DEFAULT_REFINE_PROMPT_TMPL and the input_variables list. This allows you to include a real system message in the prompt without manually adding the context and without encountering a KeyError: 'template'. The context is still managed by the vector store retriever as it is included in the input_variables list.

I hope this helps! If you have any other questions, feel free to ask.

Sources

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

dosubot[bot] commented 10 months ago

Hi, @nbrosson. I'm Dosu, and I'm helping the LangChain team manage our backlog. I wanted to let you know that we are marking this issue as stale.

Based on my understanding, the issue is about building a prompt that includes a system message, context data from a vector store, and a final question. I provided a detailed response with code examples and suggestions on how to modify the LangChain code to include a real system message in the prompt without encountering a KeyError. However, there was some uncertainty expressed by you, @nbrosson, about whether the suggested solution would help.

Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LangChain repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding and contribution to the LangChain project. If you have any further questions or concerns, please let us know.

langchain-ai / langchain