[Question]: Index creation without LLM

DSOBoy commented 11 months ago

Question Validation

[X] I have searched both the documentation and discord for an answer.

Question

I am looking for a method of retrieving context information from a set of documents and returning either the most relevant document nodes or the wrapped prompt with the query and context included.

The issue I am currently having is constructing the index without using an LLM. I have been using VectorStoreIndex with a local embedding model. Is there a method of running this without hitting the LLM authentication error.

Thank you.

dosubot[bot] commented 11 months ago

🤖

Hi there,

Based on your question, it seems you're trying to retrieve context information from a set of documents and return the most relevant document nodes or the wrapped prompt with the query and context included, using VectorStoreIndex with a local embedding model, without encountering the LLM authentication error.

To achieve this, you can use the VectorIndexRetriever class from the llama_index.indices.vector_store.retrievers.retriever.py file. Here is a step-by-step guide:

Initialize the VectorIndexRetriever with the required parameters. The index parameter should be an instance of VectorStoreIndex with a local embedding model.

retriever = VectorIndexRetriever(
    index=vector_store_index_instance,  # VectorStoreIndex instance with local embedding model
    similarity_top_k=10,  # number of top k results to return
    vector_store_query_mode=VectorStoreQueryMode.DEFAULT,  # vector store query mode
    filters=None,  # metadata filters, defaults to None
    alpha=None,  # weight for sparse/dense retrieval, only used for hybrid query mode
    node_ids=None,  # list of nodes to constrain search
    doc_ids=None,  # list of documents to constrain search
    sparse_top_k=None,  # number of top k results to return for sparse retrieval
)

Create a QueryBundle object with your query and context.

query_bundle = QueryBundle(
    query_str="your query string",
    context_str="your context string",
    embedding=None,  # embedding of the query, if available
    embedding_strs=[],  # list of strings to generate the embedding, if embedding is not available
)

Use the _retrieve or _aretrieve method of the VectorIndexRetriever to retrieve the most relevant document nodes.

nodes_with_scores = retriever._retrieve(query_bundle)

or

nodes_with_scores = await retriever._aretrieve(query_bundle)

The _retrieve and _aretrieve methods will first check if the query is an embedding query. If it is, and the embedding is not provided, it will generate the embedding using the local embedding model. Then, it will build a VectorStoreQuery object and use it to query the VectorStoreIndex. The query result will be used to build a list of NodeWithScore objects, which will be returned.

Regarding the LLM authentication error, it's not mentioned in the provided context. If you're encountering this error, it might be related to the setup or configuration of your LLM applications or the VectorStoreIndex. You might need to check your authentication credentials or the way you're initializing the VectorStoreIndex or the LLM applications.

If you could provide more details about the exact error message you're encountering, it would be easier to diagnose and solve the problem.

I hope this helps! Let me know if you have any other questions.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

logan-markewich commented 11 months ago

@DSOBoy just set llm=None in the service context

service_context = ServiceContext.from_defaults(llm=None, ...)

DSOBoy commented 11 months ago

Thanks, Working now. Appreciate it.

run-llama / llama_index