SPIKE | As a user, I want to preserve long context (better?)

[kjpap]

This issue is for a: (mark with an x)

[ ] bug report -> please search issues before submitting
[x] feature request
[ ] documentation issue or request
[ ] regression (a behavior that used to work and stopped in a new release)

Description

As a user, I want to preserve long context so that my conversation is relevant and delivers the best quality results.

Because of the token limit, conversations are constrained. To preserve context over longer conversations, a mechanism like summarization needs to be used so that the conversation provides high value.

Expected/desired behavior

• [ ] Determine design for how to preserve long-context in the chat (e.g. LangChain, )
• [ ] Document design & create another issue (feature request) for build

Mention any other details that might be useful

[kjpap]

The current solution does this to an extent, we may be ok with how it's implemented so could close this one.