run-llama / llama_index

LlamaIndex is a data framework for your LLM applications
https://docs.llamaindex.ai
MIT License
36.9k stars 5.28k forks source link

[Question]: OpensearchVectorStore Metadata #16438

Open ehsankf opened 1 month ago

ehsankf commented 1 month ago

Question Validation

Question

OpensearchVectorStore retrieves the embedding along with text as a meta data from the index. How to disable using meta data in the following modes:

MetadataMode.LLM
MetadataMode.EMBED

As a followup question, can we set to exclude embedding from metadata from the opensearch pipeline.

Thanks.

dosubot[bot] commented 1 month ago

To disable metadata usage in OpensearchVectorStore for MetadataMode.LLM and MetadataMode.EMBED, you can modify the index_results method in the OpensearchVectorClient class. It uses MetadataMode.NONE to exclude metadata from the content being indexed:

texts.append(node.get_content(metadata_mode=MetadataMode.NONE))

Regarding excluding embedding from metadata in the OpenSearch pipeline, embeddings are stored separately in the vector_field, so they are not included in the metadata by default. The current implementation already excludes embeddings from metadata [1].

To continue talking to Dosu, mention @dosu.