[Bug]: Using qdrant as vectorstore with RaptorRetriever

gtesei commented 3 months ago

Bug Description

I'm trying to use RaptorPack/RaptorRetriever with qdrant as vectorstore. Here's the code I'm using following https://github.com/run-llama/llama_index/blob/main/llama-index-packs/llama-index-packs-raptor/examples/raptor.ipynb .

import os 
import nest_asyncio
import openai

from llama_index.packs.raptor import RaptorPack
import qdrant_client
from llama_index.core.node_parser import SentenceSplitter
from llama_index.llms.openai import OpenAI
from llama_index.embeddings.openai import OpenAIEmbedding
from llama_index.packs.raptor import RaptorRetriever
from llama_index.core import StorageContext
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader

from llama_index.vector_stores.qdrant import QdrantVectorStore

# in the same dir this PDF should be downloaded before  
#!wget https://arxiv.org/pdf/2401.18059.pdf -O ./raptor_paper.pdf

documents = SimpleDirectoryReader(input_files=["./raptor_paper.pdf"]).load_data()

nest_asyncio.apply()

qclient = qdrant_client.QdrantClient(location='http://127.0.0.1:6333')
qvector_store = QdrantVectorStore(collection_name='test_raptor', client=qclient)

llm=OpenAI(model="gpt-4-32k-0613", temperature=0.0)
embed_model = OpenAIEmbedding(model="text-embedding-ada-002")

# This should be done once 
feed_qdrant = True 

if feed_qdrant:
    print(">> FEED vectorstore ... ")
    # >>> This works 
    raptor_pack = RaptorPack(
        documents,
        embed_model=embed_model,  # used for embedding clusters
        llm=llm,  # used for generating summaries
        vector_store=qvector_store,  # used for storage
        similarity_top_k=2,  # top k for each layer, or overall top-k for collapsed
        mode="collapsed",  # sets default mode
        transformations=[
            SentenceSplitter(chunk_size=400, chunk_overlap=50)
        ],  # transformations applied for ingestion
    )

print(">> CREATE RaptorRetriever ... ")
retriever = RaptorRetriever([],
                            embed_model=embed_model,  # used for embedding clusters
                            llm=llm,  # used for generating summaries
                            vector_store=qvector_store,  # used for storage
                            similarity_top_k=2,  # top k for each layer, or overall top-k for collapsed
                            mode="collapsed",  # sets default mode
                            ) 

# <<< This does not work 
nodes = retriever.retrieve("What baselines is raptor compared against?", mode="collapsed")
print(len(nodes))
print(nodes[0].text)

Although it is possible to feed the vectorstore, when I build the RaptorRetriever I got this exception:

>> CREATE RaptorRetriever ... 
Traceback (most recent call last):
  ...
    nodes = retriever.retrieve("What baselines is raptor compared against?", mode="collapsed")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../lib/python3.11/site-packages/llama_index/packs/raptor/base.py", line 294, in retrieve
    return asyncio.run(self.aretrieve(query_str, mode or self.mode))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../lib/python3.11/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "..../lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "..../lib/python3.11/asyncio/tasks.py", line 267, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/packs/raptor/base.py", line 309, in aretrieve
    return await self.collapsed_retrieval(query_str)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/packs/raptor/base.py", line 234, in collapsed_retrieval
    return await self.index.as_retriever(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "....lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 233, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/core/base/base_retriever.py", line 276, in aretrieve
    nodes = await self._aretrieve(query_bundle=query_bundle)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 233, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...../lib/python3.11/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 113, in _aretrieve
    return await self._aget_nodes_with_embeddings(query_bundle)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...../lib/python3.11/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 183, in _aget_nodes_with_embeddings
    query_result = await self._vector_store.aquery(query, **self._kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".....lib/python3.11/site-packages/llama_index/vector_stores/qdrant/base.py", line 714, in aquery
    response = await self._aclient.search(
                     ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'search'

Version

0.10.27

Steps to Reproduce

wget https://arxiv.org/pdf/2401.18059.pdf -O ./raptor_paper.pdf
python <code above>

Relevant Logs/Tracbacks

>> CREATE RaptorRetriever ... 
Traceback (most recent call last):
  ...
    nodes = retriever.retrieve("What baselines is raptor compared against?", mode="collapsed")
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/.../lib/python3.11/site-packages/llama_index/packs/raptor/base.py", line 294, in retrieve
    return asyncio.run(self.aretrieve(query_str, mode or self.mode))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../lib/python3.11/site-packages/nest_asyncio.py", line 30, in run
    return loop.run_until_complete(task)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/..../lib/python3.11/site-packages/nest_asyncio.py", line 98, in run_until_complete
    return f.result()
           ^^^^^^^^^^
  File "..../lib/python3.11/asyncio/futures.py", line 203, in result
    raise self._exception.with_traceback(self._exception_tb)
  File "..../lib/python3.11/asyncio/tasks.py", line 267, in __step
    result = coro.send(None)
             ^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/packs/raptor/base.py", line 309, in aretrieve
    return await self.collapsed_retrieval(query_str)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/packs/raptor/base.py", line 234, in collapsed_retrieval
    return await self.index.as_retriever(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "....lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 233, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/core/base/base_retriever.py", line 276, in aretrieve
    nodes = await self._aretrieve(query_bundle=query_bundle)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "..../lib/python3.11/site-packages/llama_index/core/instrumentation/dispatcher.py", line 233, in async_wrapper
    result = await func(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...../lib/python3.11/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 113, in _aretrieve
    return await self._aget_nodes_with_embeddings(query_bundle)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "...../lib/python3.11/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py", line 183, in _aget_nodes_with_embeddings
    query_result = await self._vector_store.aquery(query, **self._kwargs)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".....lib/python3.11/site-packages/llama_index/vector_stores/qdrant/base.py", line 714, in aquery
    response = await self._aclient.search(
                     ^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'search'

dosubot[bot] commented 3 months ago

The AttributeError: 'NoneType' object has no attribute 'search' you're encountering suggests that the QdrantVectorStore instance's client (_client or _aclient) is None when the search operation is attempted. This typically happens if the client was not properly initialized, which could be due to missing or incorrect initialization parameters, or if there's a flaw in the logic that sets up these client instances.

To address this issue, ensure the following:

Proper Initialization: When creating the QdrantVectorStore instance, make sure to either pass a valid client or aclient instance directly or provide all necessary parameters (url, api_key, and collection_name) so the class can initialize its own client instances correctly.
Initialization Logic Check: Review the code that initializes the QdrantVectorStore to confirm it's correctly handling the initialization parameters. It's crucial that this part of your code does not ignore or mishandle any errors that might occur during the initialization process.
Client Instance Checks: Consider adding checks in the methods that utilize _client or _aclient to verify these are not None before using them. This can prevent unclear errors and allow for more graceful error handling or clearer messaging when something goes wrong.

If these steps do not resolve the issue, it might be helpful to look into the QdrantVectorStore class's initialization method to ensure it's designed to handle the client setup correctly, especially in scenarios where the setup might fail or be skipped inadvertently.

Sources

llama-index-integrations/vector_stores/llama-index-vector-stores-qdrant/llama_index/vector_stores/qdrant/base.py

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

gtesei commented 3 months ago

As explained above QdrantVectorStore is properly initialized, as it is populated with the text chunks of the paper. This is the standard output:

>> FEED vectorstore ... 
Generating embeddings for level 0.
Performing clustering for level 0.
Generating summaries for level 0 with 15 clusters.
Level 0 created summaries/clusters: 15
Generating embeddings for level 1.
Performing clustering for level 1.
Generating summaries for level 1 with 3 clusters.
Level 1 created summaries/clusters: 3
Generating embeddings for level 2.
Performing clustering for level 2.
Generating summaries for level 2 with 1 clusters.
Level 2 created summaries/clusters: 1

run-llama / llama_index