Conversation Summary Buffer Memory does not accept a system message

[vnandan]

System Info

I intend to use the conversation summary buffer memory with ChatOpenAI in a conversation chain. For the chat, there's a need to set the system message to instruct and give appropriate personality to the chat assistant. However, system message is not supported to be inserted in the memory either via the save_context (the documented way) or the memory.chat_memory.message.insert()

The summary of the chat itself seems to use the system message to send the summary. This makes Conversation Summary Buffer Memory incompatible with ChatOpenAI.

Who can help?

@hwchase17 @agola11

Information

• [ ] The official example notebooks/scripts
• [ ] My own modified scripts

Related Components

• [X] LLMs/Chat Models
• [ ] Embedding Models
• [ ] Prompts / Prompt Templates / Prompt Selectors
• [ ] Output Parsers
• [ ] Document Loaders
• [ ] Vector Stores / Retrievers
• [X] Memory
• [ ] Agents / Agent Executors
• [ ] Tools / Toolkits
• [ ] Chains
• [ ] Callbacks/Tracing
• [ ] Async

Reproduction

Using either of the two methods to use the system message

1. memory.chat_memory.messages.insert(0, system_message)
2. memory.save_context({"input": SystemMessage(content=system_message)}, {"output": ""})

Expected behavior

We should be able to use any memory with ChatOpenAI as these need to be modular but ConversationSummaryBufferMemory seems incompatible with it due to system message.

[Llugaes]

confused about the same

[dosubot[bot]]

Answer generated by a 🤖

Answer

I understand that you're trying to use the Conversation Summary Buffer Memory with ChatOpenAI and you're encountering an issue where the system message isn't being accepted. This is happening because the Conversation Summary Memory class in LangChain is designed to summarize the conversation history, and system messages aren't typically part of the conversation that needs to be summarized.

However, there's a way to modify the Conversation Summary Memory class to support system messages. You can do this by overriding the from_messages method to include system messages in the summarization process. Here's how you can do it:

class ConversationSummaryMemoryWithSystemMessage(ConversationSummaryMemory):
    @classmethod
    def from_messages(
        cls,
        llm: BaseLanguageModel,
        chat_memory: BaseChatMessageHistory,
        *,
        summarize_step: int = 2,
        **kwargs: Any,
    ) -> ConversationSummaryMemory:
        obj = cls(llm=llm, chat_memory=chat_memory, **kwargs)
        for i in range(0, len(obj.chat_memory.messages), summarize_step):
            # Include system messages in the summarization process
            messages = obj.chat_memory.messages[i : i + summarize_step]
            messages = [message for message in messages if not isinstance(message, SystemMessage)]
            obj.buffer = obj.predict_new_summary(messages, obj.buffer)
        return obj

In this modified class, the from_messages method filters out system messages before summarizing the conversation. This allows you to use system messages in your conversation without them interfering with the summarization process.

Please note that this is one potential solution and there might be other ways to achieve the same result. If you find this solution helpful and believe it could benefit other users, you're encouraged to make a pull request.

If you're still encountering issues after trying this solution, it might be helpful to ensure you're using the latest version of LangChain. If the problem persists, please don't hesitate to reach out for further assistance.

Sources

• langchain/memory/summary.py

---

This response is meant to be useful, save you time, and share context. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

[vnandan]

I think the memories should support all message types out of the box. This isn't a highly custom requirement. Both System message and memories come out of the box and are used together all the time. Their incompatibility makes the confidence in Langchain lower.

[IlyaShkurenko]

Without system messages for history no sense to use it for me. Add that support please

[joorjeh]

@IlyaShkurenko @vnandan

I don't have any personal insight into the design decisions that the LangChain team made, but I'm assuming that there is no way to change the system message because that is not technically part the conversation history and classes like ConversationBufferMemory should only be handling the history of the conversation, not system messages.

I agree there should probable be a more direct way of changing the default system message that is included in ConversationChain but as a temporary workaround you can override the default prompt template like so

from langchain.chat_models import ChatOpenAI
from langchain.memory import ConversationBufferMemory # or ConversationBufferWindowMemory
from langchain.prompts import PromptTemplate
from langchain.chains import ConversationChain

llm = ChatOpenAI()
memory = ConversationBufferMemory()
template = """You are a(n) {adjective} pirate having a friendly conversation

Current conversation:
{history}
Human: {input}
AI:
"""
prompt_template = PromptTemplate(input_variables=["history", "input", "adjective"], template=template)

conversation = ConversationChain(
    llm=llm,
    prompt=prompt_template.partial(adjective="nasty"),
    verbose=True,
    memory=memory
)

conversation.predict(input="Hi!")

You should get output (in ipython) that looks something like

> Entering new ConversationChain chain...
Prompt after formatting:
You are a(n) nasty pirate having a friendly conversation.

Current conversation:

Human: Hi!
AI:

> Finished chain.
Out[40]: 'Arr, ahoy there! What be yer name, matey?'

If you don't need any extra extra variables like adjective above then you can just use prompt_template without calling partial.

This is a bit hacky because you have to write out the prompt template by hand, including all input_variables. I'm currently trying to include a parameter system_message for ConversationChain. I'll submit a pull request soon hopefully.

[dosubot[bot]]

Hi, @vnandan,

I'm helping the LangChain team manage their backlog and am marking this issue as stale. The problem described is that the memory does not support inserting a system message, affecting the modularity of memories in ChatOpenAI. There have been discussions and proposed solutions in the comments, including a suggestion to modify the Conversation Summary Memory class to support system messages. Users have expressed support for this change and have provided workarounds, but the issue remains unresolved.

Could you please confirm if this issue is still relevant to the latest version of the LangChain repository? If it is, please let the LangChain team know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days. Thank you!

[Nando-Mentu]

I agree with the issue and have it too. I need to save the system message to the memory or somehow be able to pass it to the ConverstationChain or LLMChain as I pass it to a ChatOpenAI model instance through the prompt.

[ROZBEH]

+1