search

run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
33.62k stars 4.72k forks source link

[Question]: Can I skip using indexing if the documents are very less? #13073

Open saidineshpola opened 2 months ago

saidineshpola commented 2 months ago

Question Validation

Question

I just want to know whether we can call the query engine in RAG with out vectorstoreindex as I have very limited data for retrieval and it will be changed at every inference call `all_sentences=['Document1 data','Document2 data'] documents = [Document(text=sent) for sent in all_sentences] self.index = VectorStoreIndex.from_documents( documents, service_context=self.service_context)

Query the index with the prompt

    query_engine = self.index.as_query_engine()`
dosubot[bot] commented 2 months ago

For your scenario with very limited and frequently changing data, you can indeed bypass using a VectorStoreIndex in RAG by leveraging alternative approaches such as a KnowledgeGraphRAGQueryEngine. This method is particularly useful as it allows for dynamic querying of a Knowledge Graph without the overhead of maintaining a vector index, making it ideal for small or frequently updated datasets.

To implement this, you would need to build or use an existing Knowledge Graph and then utilize the KnowledgeGraphRAGQueryEngine for querying. This approach dynamically generates responses based on the current data in the Knowledge Graph, thus eliminating the need for a pre-built vector index.

For more detailed guidance and examples on how to set this up, you can refer to the LlamaIndex documentation and example code provided in their repository. This will give you a clear understanding of how to leverage Knowledge Graphs for efficient information retrieval in scenarios like yours.

#### Details **Sources** - [llama-index-networks/examples/demo/contributor-2/contributor_2/app_query_engine.py](https://github.com/run-llama/llama_index/blob/main/llama-index-networks/examples/demo/contributor-2/contributor_2/app_query_engine.py) - [docs/docs/examples/query_engine/knowledge_graph_rag_query_engine.ipynb](https://github.com/run-llama/llama_index/blob/main/docs/docs/examples/query_engine/knowledge_graph_rag_query_engine.ipynb) - [llama-index-core/llama_index/core/query_engine/knowledge_graph_query_engine.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/query_engine/knowledge_graph_query_engine.py) - [llama-index-core/llama_index/core/indices/knowledge_graph/retrievers.py](https://github.com/run-llama/llama_index/blob/main/llama-index-core/llama_index/core/indices/knowledge_graph/retrievers.py) - [llama-index-packs/llama-index-packs-neo4j-query-engine/llama_index/packs/neo4j_query_engine/base.py](https://github.com/run-llama/llama_index/blob/main/llama-index-packs/llama-index-packs-neo4j-query-engine/llama_index/packs/neo4j_query_engine/base.py)
**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.