Future Research Direction Exploring 未来研究方向探讨

[Neph0s]

As a big fan of anime, I would like to make some academic contribution to the field of character cosplay, and I am currently in search of a suitable research topic related. Could you please provide offer some recommendation? I believe that the development of Chat-Haruhi has likely presented numerous challenges. I would greatly appreciate any insights or advice you might be willing to share.

[LC1332]

Oh, 我之前想在ChatHaruhi的讨论环节去多添加这样的讨论。不过很多感觉不适合在正文中出现，我来列一下：

• Chatbot的长期记忆，这个是大家都没有去很好解决的。如果希望角色和用户有一个复杂互动，长期记忆是很必要的
• 角色的记忆变化，角色在不同时期，记忆和经历是会发生变化的。现在ChatHaruhi把这个作为整体去处理，但是并不是很合适。
• 还有就是这样的训练其实最好能有小说-电影的align数据，类似去扩充Aligning books and movies: Towards story-like visual explanations by watching movies and reading books.中的数据
• 另外，对于一些保有秘密的角色，他们在对话之前需要加入一定的思考逻辑，防止泄漏秘密，这个也可以作为专门的研究

Here is the English translation:

Oh, I had previously thought about adding more discussions like this in the discussion section of arxiv paper. However, many felt it was not suitable to appear in the main text, so let me list them here:

• Long-term memory for chatbots. This is something none of us have solved well. Long-term memory is essential if we want complex interactions between the character and users.

• Changes in the character's memories. A character's memories and experiences can change during different periods. ChatHaruhi currently handles this as a whole, but it's not very suitable.

• It would also be best if we could have novel-movie aligned data for training, similar to the data in "Aligning books and movies: Towards story-like visual explanations by watching movies and reading books."

• Additionally, for characters who keep secrets, they need some logical thinking before conversations to avoid leaking secrets. This could also be an area of specialized research.

[Neph0s]

感谢您的回答。关于您提到的这些挑战，我有以下问题和粗浅的想法想和您探讨一下：

• ChatBot的长期记忆这块，随着更大context的LLM（比如使用Claude）出现，会自然得到解决吗？另外，将用户与角色的对话存入Dialogue Corpus或极端点训入模型，有尝试过这类方案吗？
• 个人好奇，目前的对话记录检索似乎是基于向量的。arxiv技术报告中似乎有提到，对话的拆分不会很精确，在对话间可能包含其他文字。这样的话，以用户问题作为query，以小说中的节选（对话）作为key的向量检索会不会存在噪音？是否需要使用BM25或其他符号算法？
• 角色的记忆变化感觉是LLM的时间感知的问题。这个我之后可以针对角色对话场景试点启发式的方案，但我觉得不太会有效（）。
• “如何不泄露秘密”是个很有意思的话题。或许可以借鉴隐私保护的相关工作？

关于小说-电影 alignment这块我没有什么想法，就不讨论了。

Here is the English translation:

Thank you for your response. I have some questions and preliminary thoughts on the challenges you mentioned, and I'd like to discuss them with you:

• Regarding the long-term memory aspect of ChatBots, will this issue naturally be resolved with the emergence of larger-context LLMs (such as using Claude)? Additionally, have there been any attempts to store user-to-role conversations in a Dialogue Corpus or train models with extreme cases?

• I'm curious about the current conversation history retrieval, which seems to be based on vectors. It appears that the splitting of dialogues may not be very precise, and there may be other text included between dialogues. In such a case, would vector retrieval using the user's question as a query and excerpts from novels (dialogues) as keys introduce noise? Would it require the use of BM25 or other symbolic algorithms?

• The changes in role memory seem to be related to the temporal awareness of LLMs. I can explore heuristic solutions for role-based dialogue scenarios later, but I don't think they will be very effective.

• "How to avoid revealing secrets" is an interesting topic. Perhaps insights from privacy protection-related work can be applied?

• I won't discuss the novel-to-movie alignment as you mentioned you don't have any thoughts on that.

[LC1332]

32K的token长度可以部分解决一些问题。为更长周期的对话任务的benchmark设计感觉还挺难的。比如要做个人助理的时候就很复杂。

对话的拆分我最近做了一个基于摘要的新版本。我感觉不是特别大的问题。但是key搜索的噪音很多，特别是连续小说中，其实相邻片段的向量都是非常相近的

角色的记忆变化可以给每个片段一个故事发生时间点，或者有树状的组织结构。这个不知道能产生什么特别的应用。

如何不泄密主要是用来做剧本杀一类的应用。

Here is the English translation:

The 32K token length can partially solve some problems. It feels quite difficult to design benchmarks for longer conversational tasks.

I recently made a new version of dialog segmentation based on summaries. I don't feel it's a particularly big problem. But there is a lot of noise in vector search, especially in continuous paragraph, where the vectors of adjacent segments are actually very similar.

Character memory changes can give each segment a story timeline, or have a tree-like organizational structure. I'm not sure if this can generate any particular applications.

Not leaking secrets is mainly used for scripted murder mystery party-type applications.

[Neph0s]

将用户对话数据存入db是否是一种长期记忆的有效方案？我看了下代码，似乎目前没有把用户对话数据存入db。

关于角色记忆变化，是否能将角色相关语料/事件按时间切分成n段，每段训练一个lora？如果用户想与角色在1.5段的记忆位置进行对话，则使用第1个lora与前1.5段的对话语料库？

剧本杀上可能也可以使用类似的，只要让该角色checkpoint自己也不知道这个秘密就好了（笑）

Here is the English version:

Is storing user dialogue data in a database an effective long-term memory solution? I looked at the code, and it seems that user dialogue data is not currently being stored in a database.

Regarding changes in role memory, would it be possible to split role-related language data/events into n segments based on time and train a separate LoRA model for each segment? If a user wants to have a conversation with the role at the 1.5 segment of memory, would they use the first LoRA with the dialogue corpus from the first 1.5 segments?

For murder mystery games, a similar approach could be used, as long as the character's checkpoint is unaware of the secret, right? (laughs)

[dinghuanghao]

关于“如何不泄露秘密”我在最近的游戏中的方法和楼上的思路很像，让Chat LLM自己也不知道”秘密“就好了。通过额外的Decision LLM/ Program 来控制秘密信息的Retrieval：

1. 好感度，给每次聊天设置一个Emotion的增减，超过一定好感度才会解锁私密数据
2. 需要额外的一个KeyWord->Secret的召回机制，当用户说到了某些特定的话，会触发私密数据的召回

[LC1332]

这是个不错的想法，或者存在一个关键词的库 攒满足够的关键词才会触发