search

opensearch-project / ml-commons

ml-commons provides a set of common machine learning algorithms, e.g. k-means, or linear regression, to help developers build ML related features within OpenSearch.
Apache License 2.0
88 stars 126 forks source link

[BUG] Failed to deploy local ONNX model "gte-large-en-v1.5" with error "input mismatch" #2636

Open t83714 opened 1 month ago

t83714 commented 1 month ago

Describe the bug

With Opensearch v2.15.0, When deploying local ONNX model "gte-large-en-v1.5", I got an error of:

input mismatch, looking for: [input_ids, attention_mask, token_type_ids]

The model zip file can be downloaded from: https://github.com/magda-io/embedding_models/releases/tag/gte-large-en-v1.5

The model can be registered & downloaded properly. But when deploy, the error above will be produced.

However, a smaller version of similar model "gte-base-en-v1.5" can be deployed (and create emebedings) properly with no issue.

The working model can be found here: https://github.com/magda-io/embedding_models/releases/tag/gte-base-en-v1.5

Related component

Plugins

To Reproduce

Following the guide here: https://opensearch.org/docs/latest/ml-commons-plugin/custom-local-models/

  1. Register a model group

  2. Register a local model

    POST /_plugins/_ml/models/_register
{
"name": "gte-large-en-v1.5",
"version": "1.5.0",
"model_group_id": "wlcnb4kBJ1eYAeTMHlV6",
"description": "gte-large-en-v1.5",
"function_name": "TEXT_EMBEDDING",
"model_format": "ONNX",
"model_content_size_in_bytes": 1271086226,
"model_content_hash_value": "72e4037cd3e2a2499a3c92759159982982871a40715ca7c38f583934aa17306b",
"model_config": {
"model_type": "BertModel",
"embedding_dimension": 768,
"framework_type": "sentence_transformers"
},
"url": "https://github.com/magda-io/embedding_models/releases/download/gte-large-en-v1.5/model.zip"
}

  3. check the status of the register operation

  4. Deploy the model via endpoint POST /_plugins/_ml/models/[mode id]/_deploy

  5. check the status of the deploy operation and you will see the fail status & error

Expected behavior

Be able to successfully deploy the ONNX format model without error.

Additional Details

Plugins Running the official Docker image with no additional plugins installed.

Screenshots If applicable, add screenshots to help explain your problem.

Host/Environment (please complete the following information):

Additional context Add any other context about the problem here.

MarcinMisztalTR commented 1 month ago

Hello, Same error with "Snowflake/snowflake-arctic-embed-m" https://huggingface.co/Snowflake/snowflake-arctic-embed-m

peternied commented 1 month ago

[Triage - attendees 1 2 3 4 5] @opensearch-project/admin Could you please move this to the ml-commons repository?