[BUG] `The requested operation is not recognized by the service` from bedrock

[joshuali925]

What is the bug? Cannot call predict after creating a model that uses bedrock claude endpoint

How can one reproduce the bug? Steps to reproduce the behavior:

1. create aws account with claude enabled in bedrock
2. create iam user with this policy

   {
       "Version": "2012-10-17",
       "Statement": [
           {
               "Sid": "BedrockInvoke",
               "Effect": "Allow",
               "Action": [
                   "bedrock:InvokeModel",
                   "bedrock:InvokeModelWithResponseStream"
               ],
               "Resource": [
                   "arn:aws:bedrock:*::foundation-model/*",
                   "arn:aws:bedrock:*:<account-id>:provisioned-model/*"
               ]
           }
       ]
   }

3. disable plugins.ml_commons.only_run_on_ml_node and set plugins.ml_commons.trusted_connector_endpoints_regex to allow "^https://bedrock\\.us-east-1\\.amazonaws\\.com/.*$"
4. create ml-commons connector

   POST /_plugins/_ml/connectors/_create
   {
       "name": "Bedrock Claude Connector",
       "description": "The connector to Bedrock service for claude model",
       "version": 1,
       "protocol": "aws_sigv4",
       "parameters": {
           "region": "us-east-1",
           "service_name": "bedrock",
           "anthropic_version": "bedrock-2023-05-31",
           "endpoint": "bedrock.us-east-1.amazonaws.com",
           "auth": "Sig_V4",
           "content_type": "application/json"
       },
       "credential": {
           "access_key": "access_key",
           "secret_key": "secret_key"
       },
       "actions": [
           {
               "action_type": "predict",
               "method": "POST",
               "url": "https://bedrock.us-east-1.amazonaws.com/model/anthropic.claude-v1/invoke",
               "headers": { 
                   "content-type": "application/json",
                   "x-amz-content-sha256": "required"
               },
               "request_body": "{\"prompt\":\"${parameters.prompt}\", \"max_tokens_to_sample\":${parameters.max_tokens_to_sample}, \"temperature\":${parameters.temperature},  \"anthropic_version\":\"${parameters.anthropic_version}\" }"
           }
       ]
   }

5. create and deploy model using the connector
6. run predict

   POST /_plugins/_ml/models/<model-id>/_predict
   {
       "parameters": {
           "prompt": "Human: hello \\n Assistant:",
           "max_tokens_to_sample": 2048,
           "temperature": 0.0000001
       }
   }

7. see error

   {
       "error": {
           "root_cause": [
               {
                   "type": "status_exception",
                   "reason": "Error from remote service: {\"message\":\"The requested operation is not recognized by the service.\"}"
               }
           ],
           "type": "status_exception",
           "reason": "Error from remote service: {\"message\":\"The requested operation is not recognized by the service.\"}"
       },
       "status": 400
   }

What is the expected behavior? output of claude completions

What is your host/environment?

• OS: [e.g. iOS] amazon linux 2 x64
• Version [e.g. 22] opensearch 2.11.0
• Plugins default installed plugins

Do you have any screenshots? If applicable, add screenshots to help explain your problem.

Do you have any additional context? the same credential works using bedrock through aws python sdk. i also tried credentials from the admin role in my aws account, but still got the same error

[dhrubo-os]

From your request looks like you didn't provide session token. Could you please check this documentation?

[joshuali925]

@dhrubo-os thanks for replying. I'm using IAM user that doesn't need session token. I also tried with temporary credentials from my admin role (with the session token), but got the same error

[joshuali925]

@ylwu-amzn helped to figure out the issue, the bedrock endpoint should be https://bedrock-runtime.us-east-1.amazonaws.com/model/anthropic.claude-v1/invoke, closing. thanks