Feature/azuresearch vector support

[lucaordronneau]

Added Support for Azure AI Search as a Vector Store

Summary

This update introduces a Python module (azuresearch_vector.py) for interacting with Azure’s search services.

Key features include

• Utilization of fastembed for embedding.
• Leveraging Azure AI Search to manage metadata by setting a single vector database with a metadata type that can be ddl, documentation, or sql. This approach maintains interaction consistency and easily adapts to the existing logic.

Example with Documentation

def add_documentation(self, doc: str) -> str:
    id = deterministic_uuid(doc) + "-doc"
    document = {
        "id": id,
        "document": doc,
        "type": "doc",
        "document_vector": self.generate_embedding(doc)
    }
    self.search_client.upload_documents(documents=[document])
    return id

def get_related_documentation(self, text: str) -> List[str]:
    result = []
    vector_query = VectorizedQuery(vector=self.generate_embedding(text), fields="document_vector")

    df = pd.DataFrame(
        self.search_client.search(
            top=self.n_results_documentation,
            vector_queries=[vector_query],
            select=["id", "document", "type"],
            filter=f"type eq 'doc'",
            vector_filter_mode=VectorFilterMode.PRE_FILTER
        )
    )

    if len(df):
        result = df["document"].tolist()
    return result

Pre-PR Actions

• Tests conducted in test_vanna.py.
• Pre-commit checks completed.

[zainhoda]

Thank you @lucaordronneau ! I had to comment out the integration test until we can set up Azure AI Search on our Azure account.