Read Mail with knowledge pipeline and issue tree

[photosssa]

[Translated by GPT]

Issue Tree

The core mechanism is a tree-based issue management system. An issue should contain the following attributes:

• Who raised it
• Who it's assigned to, if applicable
• Start date
• Deadline, if applicable
• The email in which it was raised, with a link to the original email
• Summary of the issue, which could be:
  • A new task, stating the objective to be achieved
  • A raised question, needing an answer
  • Resolution of an issue, completing a task or answering a question
• Inferred status of the issue: in progress, closed, overdue, completed
• Parent issue

The knowledge maintains an issue tree, starting from a root issue (the root can be abstract, such as the existence of an organization, not a specific one); the knowledge environment provides an interface for maintaining this issue tree:

• Add new issues
• Update issues

Parse Email

Assuming we start from a certain date, scan new emails daily. For each email:

1. Input the email and retrieve the issue tree from the knowledge base.
2. LLM prompts should include: issue tree, email content, knowledge environment. LLM completes the following reasoning:
   • The email proposes a new issue, add the issue in the knowledge environment.
   • The email changes the status of an issue:
   • Reports completion of a task
   • Answers a question
   • Explicitly changes an issue's status: considered complete, needing extension, or cancellation
   • Based on reasoning results, correctly generate calls to the knowledge environment, updating the status of the issue tree.

Potential Out of Token in Reasoning:

1. Trim off issues that are closed or overdue.
2. Based on the title characteristics, if it's a reply to a certain email, locate a specific issue, trim out a subtree.
3. For lengthy email content:
   1. First method: LLM infers the email's summary, then inputs this summary for issue reasoning.
   2. Second method (preferred): Iteratively input fragments of the email content. The single LLM inference prompt then becomes: issue tree, current email summary, that segment of email content, knowledge environment:
      • Add a new method in the environment to update the current email summary.

Build Issue Tree Based on Knowledge Pipeline

1. Pipeline input: Determine if the current time is after the start time and before the next natural day, start crawling emails within the correct range.
2. Pipeline parser: Includes the computational part for preparing user prompts and several agents.
   • Computational part: Trim the issue tree, [optional: call LLM to infer and generate a summary].
   • Agent part:
   • Agent prompts: From the structured issue tree input and email content, respond with calls to the issue tree knowledge environment.
   • Input prompts: Email content or summary, trimmed issue tree.
   • Parser process: For each input email, query (trim) the current issue tree, send the email and issue tree as a user prompt to the agent, and wait for the agent's response.

issue tree

最核心的机制是树状的issue管理，一个issue应当包含以下属性：

• 谁提出来的
• 分配给谁的，如果有的话
• 起始日期
• deadline，如果有的话
• 在哪个邮件里面提出的，引用某个email的原始链接
• 这个issue的summary，有几种情况，
  • 一个新的任务，要达成什么目标
  • 提出了一个问题，需求答案
  • 解决了某个issue，完成了task或者解答了一个问题
• 推断出来的 issue的状态，进行中，关闭，超时，完成了
• parent issue

knowledge维护一个issue tree，从一个root issue出发（root可以是抽象的，比如一个组织的存在，并不是具体的）；knowledge env 提供对这个issue tree的维护接口：

• 新增issue
• 更新issue

parse email

假定从从某个起始日期开始，以每天为单位，扫描当天新增的email，对每封email：

1. 输入email 和 从knowledge base获取 issue tree
2. llm提示词应当包括：issue tree， email正文， knowledge env， llm完成如下推理：
   • email正文提出了一个新的issue，在knowledge env新增issue
   • email正文改变了一个issue的状态
   • 通报完成了一个task
   • 回答了一个问题
   • 明确改变一个issue的状态：认为完成，要延期，认为要取消
   • 根据推理结果正确产生knowledge env 的调用，更新issue tree的状态

推理部分可能的out of token：

1. 裁剪掉已经关闭，超时的 issue
2. 根据标题特征，是不是对某个email的回复，定位到某个issue， 裁剪出 sub tree
3. 很长的邮件正文：
   1. 第一种方法：先llm推理email的summary，再把summary当正文输入推理issue
   2. 第二种方法（我觉得更好）：分片迭代输入email正文，单次llm推理的提示词就变成：issue tree， 当前email summary， 当段email正文，knowledge env：
      • env里面新增一个method，更新当前email summary

基于knowledge pipeline build issue tree

1. pipeline input： 判定当前时间晚于 起始时间并且早于下一个自然天，开始爬正确范围内的邮件输入
2. pipeline parser：包含准备user prompt 的计算部分，和几个agent
   • 计算部分： 裁剪issue tree，[可选的：调用llm推理生成summary]
   • agent 部分：
   • agent提示词：从输入的结构化issue tree， 和邮件正文，回复对issue tree knowledge env的调用
   • 输入提示词： email 正文或者summary，裁剪后的issue tree
   • parser的流程： 对每一个输入的email，查询（裁剪）当前issue tree，把email 和 issue tree 当作user prompt发送给agent，等待agent返回

[waterflier]

I have also supported Task/Todo management in workspace (this is a new component), so issue should be part of the state of workspace~ I think we need to discuss how to merge the implementation of this part. Regarding the parse email problem you raised, I also did a similar operation in the agent._llm_read_article function, you can refer to it. My idea is to treat email as a specific artic, and use the self-learning Agent of workflow to complete regular learn. Instead of completely based on knowledge pipline~

We still need to continue to discuss.

P.S: We recommend using English first, please use English as a priority for the next PR