dosubot[bot]
commented
10 months ago 🤖
Hey @nickjtay! Nice to meet you over here on GitHub. I'm Dosu, a friendly bot, and I'm here to help you sort out bugs, answer questions, and help you become a contributor! While we're waiting for a human maintainer, I'm here to assist. Don't hesitate to ask any questions!
Based on the error message you provided, it seems like the issue is related to the Language Model (LLM) not being properly initialized or referenced in the ServiceContext. This is likely causing the "AttributeError: _llm" when you're trying to unpickle a VectorStoreIndex and query against it using the llama_index library.
In the LlamaIndex codebase, the LLM is called in the _retrieve method of the VectorIndexAutoRetriever class. Here, self._service_context.llm is trying to access the LLM from the ServiceContext. If the LLM is not properly initialized in the ServiceContext, or if the ServiceContext is not correctly set up, it could lead to the "AttributeError: _llm" error.
To resolve this issue, ensure that the LLM is correctly initialized and referenced in the ServiceContext. If you're unpickling a VectorStoreIndex, you might need to re-initialize the ServiceContext and the LLM after unpickling, as these objects might not be correctly restored during the unpickling process.
Here's an example of how you might re-initialize the ServiceContext and the LLM:
from llama_index import set_global_service_context
from llama_index.llms import AzureOpenAI
from llama_index.embeddings import AzureOpenAIEmbedding
from llama_index import VectorStoreIndex, SimpleDirectoryReader, ServiceContext
from azure.storage.blob import BlobServiceClient, BlobClient, ContainerClient
import pickle
import io
# ... (code to set up Azure and llama_index)
index_name = "doc_idx"
blob_service_client = BlobServiceClient(account_url=f"https://{storage_account_name}.blob.core.windows.net", credential=storage_key)
blob_client = blob_service_client.get_blob_client(container=container_name, blob=index_name)
pickled_index = blob_client.download_blob().readall()
index = pickle.loads(pickled_index)
# Re-initialize the ServiceContext and the LLM
service_context = ServiceContext(
llm=AzureOpenAI(),
embedding_model=AzureOpenAIEmbedding(),
)
set_global_service_context(service_context)
query = "Who is widget company? What was our pitch to them?"
query_engine = index.as_query_engine()
answer = query_engine.query(query)Please try this and let me know if it resolves your issue. If not, we can continue to investigate further.
Sources
- llama_index/indices/vector_store/retrievers/auto_retriever/auto_retriever.py
- llama_index/callbacks/simple_llm_handler.py
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.
Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.
If you want to continue the conversation, start your reply with @dosu-bot.
logan-markewich
nickjtay
Bug Description
I placed the code below in full to make it easy to understand what I did in order to replicate the problem. The first snippet is the loading, indexing, and storing. The second snippet is reloading the index into memory and querying against it.
I wrote the script to create an index and store as a pickle file to azure blob storage, so I could retrieve it from the front end app when the user sends a prompt.
Here is that code:
To test it, I used the same notebook I used to pickle and store the index, but I reset the kernel. Next, I unpickled the vector store index and reinstantiated the LLM, before querying.
Here's that code:
This returned the error message I reported:
Version
Current
Steps to Reproduce
Running the first snippet of code, then resetting your kernel and running the second snippet of code should replicate the problem. Other alternatives would also be helpful, such as how to store an index to Azure storage and retrieve it later to be used by Azure Open AI in the front end app.
Relevant Logs/Tracbacks