Open surendransuri opened 4 months ago
You can't do this easily using merge skill because /document/metadata_storage_name is not an array. Please consider using a custom skill to do this concatenation
Can you please provide sample code for this. For using the custom skill is it required to have model/endpoint in Azure?
I am trying to load the split the documents into chunks and create embeddings from that. To save some contextual information I want to concatenate the file name of the each document to its chunk at the starting of the chunk.
I want to have like 'chunk: title: filename, chunk_data' in the field of chunk. how to achieve this. Normal text itself it is getting failed. Please help me resolve this
merge_skill = MergeSkill( description="Skill to Merge text data via Skillset", context= "/document/pages/", insertPreTag= " ", insertPostTag= " ", inputs=[
InputFieldMappingEntry(name="text", source="/document/pages/"), InputFieldMappingEntry(name="itemsToInsert", source="/document/metadata_storage_name"), InputFieldMappingEntry(name="span", source=["0"]), ],
outputs=[
OutputFieldMappingEntry(name="mergedText", target_name="pageContent")
],
)