dosubot[bot]
commented
8 months ago Hi, @jon-chuang! I'm Dosu, and I'm helping the LlamaIndex team manage their backlog. I wanted to let you know that we are marking this issue as stale.
From what I understand, you opened this issue suggesting the addition of a feature to the QnA Summarization model that can extract relevant information from a large context. The proposed solution involves using a local model or a language model to reduce latency, potentially addressing the issue of variable information density. However, there hasn't been any activity or comments on this issue yet.
Before we close this issue, we wanted to check with you if it is still relevant to the latest version of the LlamaIndex repository. If it is, please let us know by commenting on the issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.
Thank you for your contribution, and we look forward to hearing from you soon!
Feature Description
Summarizer can help in cases where the information is evenly distributed in the document i.e. a large amount of context is required but the language is verbose or there are many irrelevant details. Summarization specific to the query can help.
Either cheap local model or even LLM are options; the latter for reducing latency due to large context window in RAG.
Another place where it helps is that percentile and top_k don't account for variable information density. (However, this may be solved with inter-node sub-node reranking).