[Bug]: Cannot query Nebula Graph index. Found 'PromptTemplate' instead of string.

derOlek commented 1 year ago

Bug Description

When running a Llama model locally and trying to query a nebulagraph based knowledge graph index the following error is thrown:

ValueError: Argument prompt is expected to be a string. Instead found <class 'llama_index.prompts.base.PromptTemplate'>. If you want to run the LLM on multiple prompts, use generate instead.

I am running nebulagraph inside a local docker container. The llama_index version is 0.8.53.post3. Below you can find the code to reproduce this issue.

In addition to that I noticed that no data is written to nebulagraph. The space that is given just stays empty. But thats not the problem here ;)

Version

0.8.53.post3

Steps to Reproduce

Below is the code I use:

os.environ["NEBULA_USER"] = "root"
os.environ["NEBULA_PASSWORD"] = "nebula" # default is "nebula
os.environ["NEBULA_ADDRESS"] = "127.0.0.1:9669" # assumed we have NebulaGraph installed locally

def loadData():
    WikipediaReader = download_loader("WikipediaReader")
    loader = WikipediaReader()
    documents = loader.load_data(pages=['Guardians of the Galaxy Vol. 3'], auto_suggest=False)

    index = KnowledgeGraphIndex.from_documents(
        documents,
        storage_context=storage_context,
        max_triplets_per_chunk=2,
        service_context=service_context,
        space_name=space_name,
        edge_types=edge_types,
        rel_prop_names=rel_prop_names,
        tags=tags,
        include_embeddings=True,
    )

    return index

if __name__ == "__main__":
    #Models
    embedding_llm  = HuggingFaceEmbeddings(model_name="sentence-transformers/all-mpnet-base-v2")
    embed_model = LangchainEmbedding(embedding_llm)

    space_name = "test"
    edge_types, rel_prop_names = ["relationship"], ["relationship"] # default, could be omit if create from an empty kg
    tags = ["entity"] # default, could be omit if create from an empty kg
    graph_store = NebulaGraphStore(space_name=space_name, edge_types=edge_types, rel_prop_names=rel_prop_names, tags=tags)
    storage_context = StorageContext.from_defaults(graph_store=graph_store)

    llm_predictor = LlamaCpp(model_path="C:/Wizard-Vicuna-13B-Uncensored.gguf.bin", n_threads=4, n_gpu_layers=-1, temperature=0, verbose=True, n_ctx=2048)

    service_context = ServiceContext.from_defaults(
        llm_predictor=llm_predictor,
        embed_model=embedding_llm
    )
    set_global_service_context(service_context)

    index = loadData()

    query_engine = index.as_query_engine(
        include_text=True, 
        response_mode="tree_summarize",
        embedding_mode='hybrid',
        similarity_top_k=5
    )
    response = query_engine.query("Tell me about Starlord") #CRASHES HERE
    print(response)

Relevant Logs/Tracbacks

Traceback (most recent call last):
  File "C:\nebula-test.py", line 117, in <module>
    response = query_engine.query("Tell me about Starlord")
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\indices\query\base.py", line 32, in query
    return self._query(str_or_query_bundle)
    nodes = self.retrieve(query_bundle)
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\query_engine\retriever_query_engine.py", line 128, in retrieve
    nodes = self._retriever.retrieve(query_bundle)
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\indices\base_retriever.py", line 34, in retrieve
    return self._retrieve(str_or_query_bundle)
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\indices\knowledge_graph\retrievers.py", line 772, in _retrieve
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\indices\knowledge_graph\retrievers.py", line 711, in _retrieve_keyword
    entities = self._get_entities(query_bundle.query_str)
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\indices\knowledge_graph\retrievers.py", line 576, in _get_entities
    entities = self._process_entities(
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\llama_index\indices\knowledge_graph\retrievers.py", line 507, in _process_entities
    response = self._service_context.llm_predictor.predict(
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\langchain\llms\base.py", line 910, in predict
    return self(text, stop=_stop, **kwargs)
  File "C:\Users\Ole\anaconda3\envs\aitest\lib\site-packages\langchain\llms\base.py", line 864, in __call__
    raise ValueError(
ValueError: Argument `prompt` is expected to be a string. Instead found <class 'llama_index.prompts.base.PromptTemplate'>. If you want to run the LLM on multiple prompts, use `generate` instead.

dosubot[bot] commented 1 year ago

🤖

Hello @derOlek,

I'm here to assist you with your issue. I'm an AI bot designed to help solve bugs, answer questions, and guide you in becoming a contributor while you're waiting for a human maintainer.

Based on the error message you provided, it seems that the prompt argument is expected to be a string, but instead, an instance of PromptTemplate is being found. This is likely due to the way the query method of the KnowledgeGraphQueryEngine class is being used.

In your code, you're using the query method like this:

response = query_engine.query("Tell me about Starlord") #CRASHES HERE

The query method is expecting a string as an argument. If you're passing a PromptTemplate instance instead of a string, you'll encounter the ValueError you're seeing.

There was a similar issue in the LlamaIndex repository: [Documentation]: custom LLM PromptTemplate no longer needed. This issue was resolved by a pull request: support both str and templates for query_wrapper_prompt in HF LLMs. The solution was to support both strings and PromptTemplate instances for the query_wrapper_prompt in Hugging Face LLMs.

In your case, you might want to ensure that you're passing a string to the query method. If you're still encountering the ValueError, please check that the variables you're providing to the query method are indeed strings. If they're not, you'll need to convert them to strings before passing them to the method.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Best, Dosu

Sources

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

Disiok commented 1 year ago

Make sure you use the LlamaCPP and HuggingFaceEmbedding from llama-index, and also pass LlamaCpp as llm instead of llm_predictor into the service context.

run-llama / llama_index