[Backend] Consider how to summarize docs

[gatheluck]

Why

LangChain has useful function load_summarize_chain which enable to summarize long docs. But we don't know if it is possible to specify format of summary. So in this ticket, we check if it is possible and if not, we need to consider detail process for summarization.

Definition of Done

The strategy how to summarize docs is decided.

How

• Check if it is possible to specify format of summary by using load_summarize_chain function.
• If not, consider multiple options. For instance
  • option 1. Firstly, summarize docs by using load_summarize_chain function. Secondly convert format based on the summary.
  • option 2. Use raw OpenAI API
  • etc.
• Quick try for each options and share results to team, and decide the way to go.

---

Link

• Index-related chains
• 長文から論点を抽出して、その論点を軸に文章の要約を試みる

[gatheluck]

There are some methods to summarize text in LangChain

Stuffing

• Pros: Only makes a single call to the LLM. When generating text, the LLM has access to all the data at once.

• Cons: Most LLMs have a context length, and for large documents (or many documents) this will not work as it will result in a prompt larger than the context length.

Map Reduce

• Pros: Can scale to larger documents (and more documents) than StuffDocumentsChain. The calls to the LLM on individual documents are independent and can therefore be parallelized.

• Cons: Requires many more calls to the LLM than StuffDocumentsChain. Loses some information during the final combined call.

Refine

• Pros: Can pull in more relevant context, and may be less lossy than MapReduceDocumentsChain.

• Cons: Requires many more calls to the LLM than StuffDocumentsChain. The calls are also NOT independent, meaning they cannot be paralleled like MapReduceDocumentsChain. There is also some potential dependencies on the ordering of the documents.

Map-Rerank

• Pros: Similar pros as MapReduceDocumentsChain. Requires fewer calls, compared to MapReduceDocumentsChain.

• Cons: Cannot combine information between documents. This means it is most useful when you expect there to be a single simple answer in a single document.