Azure / azure-search-vector-samples

A repository of code samples for Vector search capabilities in Azure AI Search.
https://azure.microsoft.com/products/search
MIT License
690 stars 285 forks source link

Bug : Unable to create merge skill from the skillset #174

Open surendransuri opened 4 months ago

surendransuri commented 4 months ago

I am trying to load the split the documents into chunks and create embeddings from that. To save some contextual information I want to concatenate the file name of the each document to its chunk at the starting of the chunk.

I want to have like 'chunk: title: filename, chunk_data' in the field of chunk. how to achieve this. Normal text itself it is getting failed. Please help me resolve this

merge_skill = MergeSkill( description="Skill to Merge text data via Skillset", context= "/document/pages/", insertPreTag= " ", insertPostTag= " ", inputs=[
InputFieldMappingEntry(name="text", source="/document/pages/
"), InputFieldMappingEntry(name="itemsToInsert", source="/document/metadata_storage_name"), InputFieldMappingEntry(name="span", source=["0"]), ],
outputs=[
OutputFieldMappingEntry(name="mergedText", target_name="pageContent")
],
)

mattgotteiner commented 4 months ago

You can't do this easily using merge skill because /document/metadata_storage_name is not an array. Please consider using a custom skill to do this concatenation

surendransuri commented 4 months ago

Can you please provide sample code for this. For using the custom skill is it required to have model/endpoint in Azure?