search

run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
33.39k stars 4.68k forks source link

[Question]: AzureAISearchVectorStore and Advanced Retrieval Techniques #11808

Closed GiannisPikoulis closed 3 weeks ago

GiannisPikoulis commented 3 months ago

Question Validation

Question

Hello.

I was wondering whether the AzureAISearchVectorStore can be used as part of an advanced retrieval technique such as AutoRetrieval, RecursiveRetrieval, SmallToBigRetrieval, etc. I have searched for examples of such an implementation but these kind of retrievers are arguably poorly documented. Thanks in advance.

dosubot[bot] commented 3 months ago

Hey @GiannisPikoulis, nice to see you back here diving into the deep end of things!

Yes, the AzureAISearchVectorStore can indeed be used as part of advanced retrieval techniques such as AutoRetrieval, RecursiveRetrieval, SmallToBigRetrieval in the LlamaIndex framework.

The LlamaIndex framework provides various search modes (SPARSE, HYBRID, SEMANTIC_HYBRID) that facilitate different retrieval strategies. These modes leverage both traditional keyword search and semantic vector search capabilities, which are essential for implementing advanced retrieval techniques.

For AutoRetrieval, you could use the AzureQueryResultSearchSparse class for keyword-based retrieval or the AzureQueryResultSearchDefault class for vector-based retrieval, depending on whether the query string or query embedding is provided.

For RecursiveRetrieval, you could leverage the hybrid search modes (AzureQueryResultSearchHybrid or AzureQueryResultSearchSemanticHybrid) to perform an initial retrieval based on vector similarity and then refine the search results recursively with additional criteria or filters.

For SmallToBigRetrieval, you could also benefit from the hybrid search capabilities, starting with a narrow search scope (e.g., using a sparse search with a specific query string) and progressively expanding the search scope by incorporating semantic vector search to find more broadly related items.

The implementation of these classes and their methods, such as _create_query_vector for creating vector queries and _create_search_query for creating text-based search queries, provides the necessary functionality to support these advanced retrieval techniques within the LlamaIndex framework.

You can find more details in the AzureAISearchVectorStore implementation in the LlamaIndex repository.

The specific methods and classes in the LlamaIndex codebase that are used to implement the AzureAISearchVectorStore are:

These are imported from llama_index.vector_stores.azureaisearch.base and are explicitly made available for external use as indicated by their inclusion in the __all__ list in the module's __init__.py file. You can find more details in the llama-index-integrations/vector_stores/llama-index-vector-stores-azureaisearch/llama_index/vector_stores/azureaisearch/init.py file in the LlamaIndex repository.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

#### About Dosu This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.
logan-markewich commented 3 months ago

It sure can. AzureAISearchVectorStore is just another vector store. Any retrieval techniques that work with other vector dbs should work with azure ai vector search.