Compatibility issue between Qdrant and DSPy when Qdrant is used as the VectorStoreIndex's storage context

[tituslhy]

Bug Description

I was following the llamaindex-dspy cookbook with the only differences being that my VectorStoreIndex object was using LlamaIndex's QdrantVectorStore with bm42 enabled as my storage context and my input files are different. When trying to compile my teleprompter, I ran into the error:

AttributeError: 'QdrantVectorStore' object has no attribute '_client'

Version

0.10.55

Steps to Reproduce

Here's a snippet of my code:

import qdrant_client
from llama_index.vector_stores.qdrant import QdrantVectorStore
from llama_index.core import Settings, StorageContext
from llama_index.core.query_pipeline import (
    QueryPipeline as QP,
    InputComponent,
    FnComponent
)
from llama_index.core.evaluation import SemanticSimilarityEvaluator
from dspy.teleprompt import BootstrapFewShot
from dspy.predict.llamaindex import (
    DSPyComponent,
    LlamaIndexModule
)

client = qdrant_client.QdrantClient("http://localhost:6333")
aclient = qdrant_client.AsyncQdrantClient("http://localhost:6333")
vector_store = QdrantVectorStore(
    collection_name="my_collection",
    client=client,
    aclient=aclient,
    fastembed_sparse_model="Qdrant/bm42-all-minilm-l6-v2-attentions",
)
storage_context = StorageContext.from_defaults(vector_store=vector_store)
docs = SimpleDirectoryReader(input_files=[...])
index = VectorStoreIndex.from_documents(
    docs,
    storage_context=storage_context
)
retriever = index.as_retriever(similarity_top_k=4, sparse_top_k=10)

dspy_component = DSPyComponent(
    dspy.ChainOfThought(GenerateAnswer)
)
retriever_post = FnComponent(
    lambda contexts: "\n\n".join([n.get_content() for n in contexts])
)
p = QP(verbose=True)
p.add_modules(
    {
        "input": InputComponent(),
        "retriever": retriever,
        "retriever_post": retriever_post,
        "synthesizer": dspy_component,
    }
)
p.add_link("input", "retriever")
p.add_link("retriever", "retriever_post")
p.add_link("input", "synthesizer", dest_key="query_str")
p.add_link("retriever_post", "synthesizer", dest_key="context_str")

dspy_qp = LlamaIndexModule(p)

train_examples=[...] #list of dspy-ai example objects with query string and answer
train_examples = [t.with_inputs("query_str") for t in train_examples]
teleprompter = BootstrapFewShot(
    max_labeled_demos=0,
    metric=validate_context_and_answer
)
compiled_dspy_qp = teleprompter.compile(dspy_qp, trainset=train_examples)

Relevant Logs/Tracbacks

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[22], line 5
      1 teleprompter = BootstrapFewShot(
      2     max_labeled_demos=0,
      3     metric=validate_context_and_answer
      4 )
----> 5 compiled_dspy_qp = teleprompter.compile(dspy_qp, trainset=train_examples)

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/dspy/teleprompt/bootstrap.py:84, in BootstrapFewShot.compile(self, student, teacher, trainset)
     82 self._prepare_student_and_teacher(student, teacher)
     83 self._prepare_predictor_mappings()
---> 84 self._bootstrap()
     86 self.student = self._train()
     87 self.student._compiled = True

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/dspy/teleprompt/bootstrap.py:154, in BootstrapFewShot._bootstrap(self, max_bootstraps)
    151     break
    153 if example_idx not in bootstrapped:
--> 154     success = self._bootstrap_one_example(example, round_idx)
    156     if success:
    157         bootstrapped[example_idx] = True

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/dspy/teleprompt/bootstrap.py:210, in BootstrapFewShot._bootstrap_one_example(self, example, round_idx)
    208         current_error_count = self.error_count
    209     if current_error_count >= self.max_errors:
--> 210         raise e
    211     dspy.logger.error(f"Failed to run or to evaluate example {example} with {self.metric} due to {e}.")
    213 if success:

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/dspy/teleprompt/bootstrap.py:190, in BootstrapFewShot._bootstrap_one_example(self, example, round_idx)
    187     predictor_cache[name] = predictor.demos
    188     predictor.demos = [x for x in predictor.demos if x != example]
--> 190 prediction = teacher(**example.inputs())
    191 trace = dsp.settings.trace
    193 for name, predictor in teacher.named_predictors():

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/dspy/primitives/program.py:26, in Module.__call__(self, *args, **kwargs)
     25 def __call__(self, *args, **kwargs):
---> 26     return self.forward(*args, **kwargs)

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/dspy/predict/llamaindex.py:261, in LlamaIndexModule.forward(self, **kwargs)
    259 def forward(self, **kwargs: Any) -> Dict[str, Any]:
    260     """Forward."""
--> 261     output_dict = self.query_pipeline.run(**kwargs, return_values_direct=False)
    262     return dspy.Prediction(**output_dict)

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/query_pipeline/query.py:410, in QueryPipeline.run(self, return_values_direct, callback_manager, batch, *args, **kwargs)
    406     query_payload = json.dumps(str(kwargs))
    407 with self.callback_manager.event(
    408     CBEventType.QUERY, payload={EventPayload.QUERY_STR: query_payload}
    409 ) as query_event:
--> 410     outputs, _ = self._run(
    411         *args,
    412         return_values_direct=return_values_direct,
    413         show_intermediates=False,
    414         batch=batch,
    415         **kwargs,
    416     )
    418     return outputs

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/query_pipeline/query.py:774, in QueryPipeline._run(self, return_values_direct, show_intermediates, batch, *args, **kwargs)
    772     return result_outputs, intermediates
    773 else:
--> 774     result_outputs, intermediates = self._run_multi(
    775         {root_key: kwargs}, show_intermediates=show_intermediates
    776     )
    778     return (
    779         self._get_single_result_output(result_outputs, return_values_direct),
    780         intermediates,
    781     )

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/query_pipeline/query.py:925, in QueryPipeline._run_multi(self, module_input_dict, show_intermediates)
    923 if self.verbose:
    924     print_debug_input(module_key, module_input)
--> 925 output_dict = module.run_component(**module_input)
    927 if (
    928     show_intermediates
    929     and module_key not in run_state.intermediate_outputs
    930 ):
    931     run_state.intermediate_outputs[module_key] = ComponentIntermediates(
    932         inputs=module_input, outputs=output_dict
    933     )

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/base/query_pipeline/query.py:203, in QueryComponent.run_component(self, **kwargs)
    201 kwargs.update(self.partial_dict)
    202 kwargs = self.validate_component_inputs(kwargs)
--> 203 component_outputs = self._run_component(**kwargs)
    204 return self.validate_component_outputs(component_outputs)

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/base/base_retriever.py:345, in RetrieverComponent._run_component(self, **kwargs)
    343 def _run_component(self, **kwargs: Any) -> Any:
    344     """Run component."""
--> 345     output = self.retriever.retrieve(kwargs["input"])
    346     return {"output": output}

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py:230, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    226 self.span_enter(
    227     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    228 )
    229 try:
--> 230     result = func(*args, **kwargs)
    231 except BaseException as e:
    232     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/base/base_retriever.py:243, in BaseRetriever.retrieve(self, str_or_query_bundle)
    238 with self.callback_manager.as_trace("query"):
    239     with self.callback_manager.event(
    240         CBEventType.RETRIEVE,
    241         payload={EventPayload.QUERY_STR: query_bundle.query_str},
    242     ) as retrieve_event:
--> 243         nodes = self._retrieve(query_bundle)
    244         nodes = self._handle_recursive_retrieval(query_bundle, nodes)
    245         retrieve_event.on_end(
    246             payload={EventPayload.NODES: nodes},
    247         )

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/instrumentation/dispatcher.py:230, in Dispatcher.span.<locals>.wrapper(func, instance, args, kwargs)
    226 self.span_enter(
    227     id_=id_, bound_args=bound_args, instance=instance, parent_id=parent_id
    228 )
    229 try:
--> 230     result = func(*args, **kwargs)
    231 except BaseException as e:
    232     self.event(SpanDropEvent(span_id=id_, err_str=str(e)))

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py:101, in VectorIndexRetriever._retrieve(self, query_bundle)
     95     if query_bundle.embedding is None and len(query_bundle.embedding_strs) > 0:
     96         query_bundle.embedding = (
     97             self._embed_model.get_agg_embedding_from_queries(
     98                 query_bundle.embedding_strs
     99             )
    100         )
--> 101 return self._get_nodes_with_embeddings(query_bundle)

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/core/indices/vector_store/retrievers/retriever.py:177, in VectorIndexRetriever._get_nodes_with_embeddings(self, query_bundle_with_embeddings)
    173 def _get_nodes_with_embeddings(
    174     self, query_bundle_with_embeddings: QueryBundle
    175 ) -> List[NodeWithScore]:
    176     query = self._build_vector_store_query(query_bundle_with_embeddings)
--> 177     query_result = self._vector_store.query(query, **self._kwargs)
    178     return self._build_node_list_from_query_result(query_result)

File /App/tlim2/Anaconda3/envs/llamaindex/lib/python3.12/site-packages/llama_index/vector_stores/qdrant/base.py:799, in QdrantVectorStore.query(self, query, **kwargs)
    795     return self.parse_to_query_result(sparse_response[0])
    797 elif self.enable_hybrid:
    798     # search for dense vectors only
--> 799     response = self._client.search_batch(
    800         collection_name=self.collection_name,
    801         requests=[
    802             rest.SearchRequest(
    803                 vector=rest.NamedVector(
    804                     name=DENSE_VECTOR_NAME,
    805                     vector=query_embedding,
    806                 ),
    807                 limit=query.similarity_top_k,
    808                 filter=query_filter,
    809                 with_payload=True,
    810             ),
    811         ],
    812     )
    814     return self.parse_to_query_result(response[0])
    815 else:

AttributeError: 'QdrantVectorStore' object has no attribute '_client'

[dosubot[bot]]

To resolve the AttributeError: 'QdrantVectorStore' object has no attribute '_client', ensure that the QdrantVectorStore class is properly initialized with either a client, aclient, or the necessary url and api_key parameters. The _client attribute is set during initialization based on these parameters.

Here is an example of how to properly initialize the QdrantVectorStore:

import qdrant_client
from llama_index.vector_stores.qdrant import QdrantVectorStore

# Initialize with a QdrantClient instance
client = qdrant_client.QdrantClient(url="http://localhost:6333")

vector_store = QdrantVectorStore(
    collection_name="my_collection", 
    client=client
)

Alternatively, you can initialize it with the url and api_key directly:

from llama_index.vector_stores.qdrant import QdrantVectorStore

vector_store = QdrantVectorStore(
    collection_name="my_collection", 
    url="http://localhost:6333", 
    api_key="your_api_key"
)

Ensure that the client, aclient, url, and api_key parameters are correctly provided to avoid the AttributeError. The QdrantVectorStore class requires these parameters to initialize the _client and _aclient attributes, which are used for interacting with the Qdrant service. If these parameters are not provided correctly, the _client attribute will not be set, leading to the AttributeError ^[1][2].

To continue talking to Dosu, mention @dosu.

[tituslhy]

Yes @dosu it's already initialized if you were to look at my code snippet earlier:

client = qdrant_client.QdrantClient("http://localhost:6333")
aclient = qdrant_client.AsyncQdrantClient("http://localhost:6333")
vector_store = QdrantVectorStore(
    collection_name="my_collection",
    client=client,
    aclient=aclient,
    fastembed_sparse_model="Qdrant/bm42-all-minilm-l6-v2-attentions",
)

[dosubot[bot]]

Ensure that the QdrantVectorStore is initialized correctly by providing either a client or aclient, but not both simultaneously. Here is the corrected initialization:

import qdrant_client
from llama_index.vector_stores.qdrant import QdrantVectorStore

# Initialize with a QdrantClient instance
client = qdrant_client.QdrantClient(url="http://localhost:6333")

vector_store = QdrantVectorStore(
    collection_name="my_collection", 
    client=client,
    fastembed_sparse_model="Qdrant/bm42-all-minilm-l6-v2-attentions",
)

Alternatively, if you need to use the asynchronous client, initialize it as follows:

import qdrant_client
from llama_index.vector_stores.qdrant import QdrantVectorStore

# Initialize with an AsyncQdrantClient instance
aclient = qdrant_client.AsyncQdrantClient(url="http://localhost:6333")

vector_store = QdrantVectorStore(
    collection_name="my_collection", 
    aclient=aclient,
    fastembed_sparse_model="Qdrant/bm42-all-minilm-l6-v2-attentions",
)

Ensure that only one of client or aclient is provided to avoid conflicts. This should properly set the _client attribute and resolve the AttributeError ^[1].

To continue talking to Dosu, mention @dosu.

[logan-markewich]

Dspy relies on pickling things, but not everything is picklesble. Seems like a general imcompatibilify here

[tituslhy]

DSPy has a Qdrant abstraction on its own though.

https://qdrant.tech/documentation/frameworks/dspy/

[tituslhy]

Sorry I think this issue still exists - i'm still getting the same error message "AttributeError: 'QdrantVectorStore' object has no attribute '_client'". I also noticed that my baseline performance without using a vector database has also deteriorated. For some reason the llamaindex query engine has some problems answering the question once the qa_prompt_tmpl object is exported from dspy.

from dspy.predict.llamaindex import DSPyPromptTemplate

qa_prompt_tmpl = DSPyPromptTemplate(
    compiled_dspy_qp.query_pipeline.module_dict["synthesizer"].predict_module
)
query_engine = index.as_query_engine(
    text_qa_template = qa_prompt_tmpl
)
response = query_engine.query(
    "What do investors look out for in a P/E ratio?"
)

yields the answer

The query string is empty, so there is no specific question to answer. Without a question, I cannot provide a relevant response. If you have a specific question about price-to-earnings (P/E) ratios or any other financial topic, please feel free to ask and I'll be happy to help.