[ML] PUT trained model fails to set prefix string for elastic provided model

maxhniebergall commented 3 months ago

Elasticsearch Version

serverless, main, all

Installed Plugins

No response

Java Version

bundled

OS Version

serverless

Problem Description

When the PutTrainedModel action is called with the optional prefix string argument, if the model is a provided model such as .multilingual-e5-small_linux-x86_64 the provided prefix strings are incorrectly overridden with values from the model package repository. We need to change this so that the values provided during creation take precedence over those stored in the package repository.

Here is where the override happens: https://github.com/elastic/elasticsearch/blob/330e8b99bf8342ed35d8e3a3e68c68a20b087f5c/x-pack/plugin/ml-package-loader/src/main/java/org/elasticsearch/xpack/ml/packageloader/action/TransportGetTrainedModelPackageConfigAction.java#L89

Steps to Reproduce

PUT _ml/trained_models/.multilingual-e5-small_linux-x86_64 { "input": { "field_names": [ "text_field" ] }, "prefix_strings": { "ingest": "query: ", "search": "query: " } }

the response contains: "prefix_strings": { "ingest": "passage: ", "search": "query: " }

the string "passage: " doesn't appear anywhere in elasticsearch, and so must be being pulled in from the model repository metadata

Logs (if relevant)

No response

elasticsearchmachine commented 3 months ago

Pinging @elastic/ml-core (Team:ML)

maxhniebergall commented 3 months ago

Users probably don't want to update prefix strings for provided models as the strings they provide won't be the same as the ones used to train the models. We need to update the docs to make this clear. Better yet, we could try to remove the body for model put requests all together.

davidkyle commented 3 months ago

The model should not be used with arbitrary prefix strings, Elasticsearch protects against this by not overwriting the configured prefixes. In certain situations it might be valid to use the query: prefix at both ingest and search, for users who want to do that they can use Eland to install the model from HuggingFace and override the prefixes with the --ingest-prefix and --search-prefix options. The easiest way to get started with Eland is to use the docker image, first pull the image:

docker pull docker.elastic.co/eland/eland

Then install the model into your cluster:

docker run -it --rm --network host docker.elastic.co/eland/eland \
    eland_import_hub_model \
      --url 'http://HOST:9200' \
      -u ELASTIC_USER -p PASSWORD \
      --hub-model-id 'intfloat/multilingual-e5-small' \
      --task-type text_embedding \
      --search-prefix="query: " --ingest-prefix="query: "

In this example both the search and ingest prefixes are set to "query: "

elastic / elasticsearch