opensearch-project / ml-commons

ml-commons provides a set of common machine learning algorithms, e.g. k-means, or linear regression, to help developers build ML related features within OpenSearch.
Apache License 2.0
88 stars 125 forks source link

[FEATURE] Improve error message when parameters are missing in predict request #1704

Open joshuali925 opened 8 months ago

joshuali925 commented 8 months ago

Is your feature request related to a problem? Connector setup:

{
  "name": "claude connector",
  "description": "The connector for claude model",
  "version": 1,
  "protocol": "aws_sigv4",
  "parameters": {
      "region": "us-east-1",
      "service_name": "bedrock",
      "anthropic_version": "bedrock-2023-05-31",
      "endpoint": "bedrock.us-east-1.amazonaws.com",
      "auth": "Sig_V4",
      "content_type": "application/json"
  },
  "credential": {  },
  "actions": [
    {
      "action_type": "predict",
      "method": "POST",
      "url": "<endpoint>",
      "headers": {
        "content-type": "application/json",
        "x-amz-content-sha256": "required"
      },
      "request_body": "{\"prompt\":\"${parameters.prompt}\", \"max_tokens_to_sample\":${parameters.max_tokens_to_sample}, \"temperature\":${parameters.temperature},  \"anthropic_version\":\"${parameters.anthropic_version}\" }"
    }
  ]
}

Request:

curl -X POST -k -H 'Content-Type: application/json' localhost:9200/_plugins/_ml/models/28RqF4wB7w3XlCm_W0My/_predict -d '{"parameters":{"prompt":"\n\nHuman: hi\n\nAssistant:"}}'

Response:

{"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Invalid JSON in payload"}],"type":"illegal_argument_exception","reason":"Invalid JSON in payload"},"status":400}%

But Invalid JSON in payload is not helpful because the request payload contains a valid JSON.

What solution would you like? Show a better error message in https://github.com/opensearch-project/ml-commons/blob/d1986404463df55850a0fa28b789c121f8b19e4a/common/src/main/java/org/opensearch/ml/common/connector/HttpConnector.java#L291-L295. Currently it throws Invalid JSON in payload because {"prompt":"\n\nHuman: hi\n\nAssistant:", "max_tokens_to_sample":${parameters.max_tokens_to_sample}, "temperature":${parameters.temperature}, "anthropic_version":"bedrock-2023-05-31" } is not valid JSON, but it should mention it's because some parameters are missing.

What alternatives have you considered? Fill non-supplied parameters with empty/default values

Do you have any additional context? I'm using feature/agent_framework_dev branch, not sure if other branches have different behaviors.

mingshl commented 3 weeks ago

@Zhangxunmt is this issue being worked on?