Generate examples and insights to be used in our upcoming research piece

[garvk]

Likely actions to be performed:

1. Identify characteristics of different RAGs in Ragstore currently, viz. Base, Subqa, Raptor etc
2. Generate examples and prove the value of aggrag
3. Use some examples to lay the foundation for the research piece

Actionable items:

• [ ] Come up with examples (let's have an initial fixed number of examples):
  • [ ] Evaluate example queries across all RAGs.
  • [ ] Filter examples which best portrays the concept of AggRAG.
  • [ ] Fine tune the queries and finalise the research piece examples.
• [ ] Come up with the flow, content, sub-sections.
• [ ] Create a first draft showcasing how RAG ensembles work.

[Munir-fractal]

1. Base: Questions from tables are considered out of context.
2. Raptor: Unable to answer cell-specific questions from any table with little exception, but answers broader questions from tables well.
3. SubQA: Provides non-cohesive answers for questions from table, able to give cell-specific answer at times.
4. TableBase: Best at providing cell-specific answers, not so good at broader questions.

[garvk]

Descriptions for 3 RAGs

Base: Nothing fancy, foundational RAG that generates context-aware responses by retrieving relevant information from a given document set. Leverages llama-index; using SimpleDirectoryReader for document loading and VectorStoreIndex for index creation. Streaming and chat history is allowed, but currently disabled.

Pro tip: answers general questions from the document which are directly referenced. Little to no insight is provided more than the source material

SubQA: More power to user queries. Breaks down a user query into smaller relevant queries. Each query generates context-aware responses, and the responses are combined into a single comprehensive answer. Streaming is not possible. Chat history is possible, but currently disabled. Click here for more info<Insert link: https://docs.llamaindex.ai/en/stable/examples/output_parsing/guidance_sub_question/>

Pro tip: provides a wider description of concepts/facts provided in the document, even beyond the scope of the document but limited to the context.

Raptor: Consolidates info context into clear summaries. Adapts a bottom-up approach by clustering information chunks to form a hierarchical tree structure. Click here for more info<Insert link: https://docs.llamaindex.ai/en/stable/api_reference/packs/raptor/>

Pro tip: Best at answering factual questions which requires more specific prompts. Prefer this when numbers or values are expected.

[tarun-goyal]

@Munir-fractal to move this ticket to "In Review" after creating sub-tasks on this.

[Munir-fractal]

Examples:

rag_analysis-examples.pdf

AI_domain_eg.pdf