search

getmetal / motorhead

🧠 Motorhead is a memory and information retrieval server for LLMs.
https://getmetal.io
Apache License 2.0
842 stars 79 forks source link

Messages summarized in wrong order #23

Closed soylomass closed 1 year ago

soylomass commented 1 year ago

Messages are passed in the wrong order to the summarizer. Example:

messages_joined: AI: If you have any questions or if there's anything else you'd like to talk about, feel free to let me know! Human: no AI: Alright then. If you have any questions or if there's anything else you'd like to talk about, feel free to let me know! Human: no AI: Is there something that you would like to ask me or discuss? I'm here to help in whatever way I can. Human: no AI: Hello! How can I assist you today?

In this example, the last one was the first message.

In many circumstances, this doesn't matter, but sometimes it does. In a similar conversation, the summary returned:

The human initially rejects the AI's offer of assistance, but the AI persists in asking how it can help. The human eventually greets the AI, to which the AI responds with an inquiry...

This summary is wrong, as the human didn't "finally greet", but started the conversation greeting.

soylomass commented 1 year ago

Seems that the messages are put into the prompt in the wrong order when using langchainjs and the motorhead memory too. Can anyone confirm? PS: I'm using returnMessages: true mode

Edit: I temporarily solved this issue by reversing the messages after they are retrieved from Redis (in both memory.rs and reducer.rs). Not sure if I broke something by doing that but it's working correctly now.

Czechh commented 1 year ago

@soylomass confirmed, working on a fix for the summarizer portion.

Czechh commented 1 year ago

Thank you for the issue! Should be fixed now 🤘