Summary does not reflect all history if you ask for fewer than 12 messages

[JasonMaggard]

The size of our prompt was leaving our AI slow. With doc stores and history our prompts were getting into the 1,000 token range and prompt eval time got up to 2 minutes.

When I tried to limit the history to the last 5 messages, I noticed that the summary was never returned until there were 12 messages in history. When the summary is returned, it is only a summary for the messages > 12.

<s>[INST] <<SYS>>
You are cordial, professional, honest and you do not make up answers. You avoid answers that make you sound like an AI or large language model.
You don't feel the need to introduce yourself or greet someone when starting a response.

Use the following context to help answer the question if it is available and relevant to the question asked.

Here is a memory of our conversation. My questions are labeled 'human' and your responses are labeled 'ai'. Use this to help answer questions if it is relevant.
Summary: The human asks about the state of the union in America. The AI responds by stating that the State of the Union is strong and provides examples of job creation and global cooperation. The human then asks about The Witcher, and the AI explains that it is a popular Netflix show based on a fantasy book series by Andrzej Sapkowski. The AI mentions the main character, Geralt of Rivia, and the talented cast of the show. The human further asks about the love interest in the book, and the AI identifies Yennefer as Geralt's love interest and provides background information on her character. The AI also mentions the actress who portrays Yennefer in the Netflix series. The human then asks about another character, Ciri. The AI responds by giving a detailed description of Ciri, her role in the book series and the Netflix adaptation, and her relationships with Geralt and Yennefer. The AI summarizes the book series, mentioning Geralt's role as a witcher, his encounters with mythical creatures and political intrigue, and the complex relationships between the main characters. The human then inquires about the reception of the Netflix series, and the AI explains that it has generally received positive reviews for its production values, acting, and faithfulness to the source material. The AI highlights the strong portrayal of the female characters and provides examples of praise from Variety and The New York Times. However, the AI acknowledges that some reviewers have mentioned pacing issues and confusion for viewers unfamiliar with the books. Overall, the Netflix series has been widely praised for its quality and faithfulness to the books.
ai:
Ah, Freya Allan! She portrays Ciri in the Netflix series "The Witcher" and has received positive reviews for her performance. Reviewers have praised her ability to bring depth and vulnerability to the character, particularly in her portrayal of Ciri's emotional struggles and relationships with the other characters.
For example, The Telegraph wrote, "Freya Allan shines as the young princess Ciri, bringing a sense of vulnerability and determination to the role." The Independent stated, "Allan delivers a standout performance as the fierce and determined Ciri."
However, some reviewers have noted that Allan's performance may not be to everyone's taste, particularly if viewers are familiar with the book series or previous adaptations of "The Witcher". Nevertheless, her portrayal of Ciri has been widely praised for its faithfulness to the source material and her ability to bring the character to life on screen.
human: Who is the villian in this book?
ai: Ah, the villain in "The Witcher" book series! That would be the sorceress, Yennefer of Vengerberg. She is a complex and multifaceted character who plays a pivotal role in the story as both a love interest of Geralt's and a powerful sorceress in her own right.
Yennefer is driven by a desire for power and control, and she will stop at nothing to achieve her goals. She is ruthless and cunning, and will use any means necessary to gain the upper hand. However, she is also a deeply conflicted character, struggling with her own moral ambiguity and the consequences of her actions.
In the Netflix series, Yennefer is portrayed by actress Anya Chalotra, who brings depth and nuance to the character. Her performance has been widely praised for its faithfulness to the source material and her ability to bring the character to life on screen.

<</SYS>>

Question: What books are similar that I might enjoy?[/INST]
</s>

In this instance, the summary does not reflect the questions about Henry Cavill or Freya Allen being good actors nor the answers when asking for only the last three memory messages.

This is easily replicated. Just load 10 messages into history and request the last 4. There will be no summary. Thanks.

[danielchalef]

@JasonMaggard This is by design. In general, the summarizer only summarizes messages beyond the message window. More about how it operates may be found here: https://docs.getzep.com/sdk/extractors/#summarizer-extractor

Rather than limiting the number of messages returned by passing the limit parameter, you may want to reduce the message window itself.

Configuration details may be found here: https://docs.getzep.com/deployment/config/

[JasonMaggard]

I have noticed the same behaviour using the env variable and using the limit parameter. ZEP_MEMORY_MESSAGE_WINDOW

[danielchalef]

@JasonMaggard Looks like you closed the issue before I could respond. Did you successfully set the message window ?