search

elastic / eland

Python Client and Toolkit for DataFrames, Big Data, Machine Learning and ETL in Elasticsearch
https://eland.readthedocs.io
Apache License 2.0
628 stars 98 forks source link

[NLP] Add prefix_string config option to the import model hub script #642

Closed davidkyle closed 6 months ago

davidkyle commented 7 months ago

The prefix_strings option was added in https://github.com/elastic/elasticsearch/pull/102089 to support the E5 model.

The new params to eland_import_hub_model are --search-prefix and --ingest-prefix

Example usage

eland_import_hub_model       
  --url 'http://localhost:9200'
  --hub-model-id intfloat/multilingual-e5-small       
  --task-type text_embedding 
  --search-prefix "query: " 
  --ingest-prefix "passage: "

This option is only available in Elasticsearch 8.12 and depends on https://github.com/elastic/elasticsearch-specification/pull/2363