Azure / azure-search-vector-samples

A repository of code samples for Vector search capabilities in Azure AI Search.
https://azure.microsoft.com/products/search
MIT License
690 stars 285 forks source link

Issue : langchain sample fails #182

Open surendransuri opened 4 months ago

surendransuri commented 4 months ago

I tried to replicate the notebook azure-search-vector-python-langchain-sample.ipynb to add documents to the index. Here the schema for index created is not provided please provide the index schema creation code for this sample.

Example: I tried to create a manual schema like below and tried to add documents

Index Schema Creation: fields = [
SearchField(name="id", type=SearchFieldDataType.String, key=True, sortable=True, filterable=True, facetable=True, analyzer_name="keyword"),
SearchField(name="content", type=SearchFieldDataType.String, sortable=False, filterable=False, facetable=False),
SearchField(name="page_content", type=SearchFieldDataType.String, sortable=False, filterable=False, facetable=False),
ComplexField(name="metadata", fields=[SearchField(name="source", type=SearchFieldDataType.String), SearchField(name="page", type=SearchFieldDataType.Int32), ]
), SearchField(name="vector", type=SearchFieldDataType.Collection(SearchFieldDataType.Single), vector_search_dimensions=1536, vector_search_profile_name="myHnswProfile"),
]

Failed in below line of code with the mentioned error: line of code --->

results = vector_store.add_documents(documents=file_chunks)

(file_chunks structure:

[Document(page_content='title:PerksPlus.pdf, PerksPlus Health and Wellness \nReimbursement Program for \nContoso Electronics Employees' metadata={'source': 'C:/Users/Surs092/Downloads/Data\PerksPlus.pdf', 'page': 0} ]

)

Error message: Exception Occured () The request is invalid. Details: A 'PrimitiveValue' node with non-null value was found when trying to read the value of the property 'metadata'; however, a 'StartArray' node, a 'StartObject' node, or a 'PrimitiveValue' node with null value was expected. Code: Message: The request is invalid. Details: A 'PrimitiveValue' node with non-null value was found when trying to read the value of the property 'metadata'; however, a 'StartArray' node, a 'StartObject' node, or a 'PrimitiveValue' node with null value was expected. for file PerksPlus.pdf Indexed 0 chunks

Also, please provide sample code to create and add custom fields to the vector index

mattgotteiner commented 4 months ago

Thanks, we'll try to reproduce this issue

roflaala commented 1 month ago

This issue got resolved for me when I made a change in site-packages\langchain_community\vectorstores\azuresearch.py file. I changed FIELDS_METADATA: json.dumps(metadata) to FIELDS_METADATA: metadata on line 348.