InvokeAgentCommand doesn't return chunk

davidgamify commented 2 weeks ago

Checkboxes for prior research

[X] I've gone through Developer Guide and API reference
[X] I've checked AWS Forums and StackOverflow.
[X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

When I call the InvokeAgentCommand send function, I receive a response, but the response doesn't include the chunk. When I use the same parameters in python3 (boto3), I get chunk in the result.

Regression Issue

[ ] Select this option if this issue appears to be a regression.

SDK version number

"@aws-sdk/client-bedrock-agent-runtime": "^3.658.0"

Which JavaScript Runtime is this issue in?

React Native

Details of the browser/Node.js/ReactNative version

v22.7.0

Reproduction Steps

  const testInput = {
    agentId, // required
    agentAliasId, // required
    sessionId: _sessionId, // required
    inputText: question,
  };

  const command = new InvokeAgentCommand(testInput);
  const response = await client.send(command);
  console.log ("!!!response: ", response);
  let completion = ""
  for await (const chunkEvent of response.completion) {
    if (chunkEvent.chunk) {
      const chunk = chunkEvent.chunk;
      let decoded = new TextDecoder("utf-8").decode(chunk.bytes);
      completion += decoded;
    }
  }
  LOG.debug("!!!completion: ", completion);

Observed Behavior

!!!response in log =

{
  "$metadata": {
    "httpStatusCode": 200,
    "requestId": "ae975225-b133-42c4-a8eb-0d425e650a40",
    "attempts": 1,
    "totalRetryDelay": 0
  },
  "contentType": "application/json",
  "sessionId": "96c1843f-8f91-4ff8-8742-753c4a0dbf91",
  "completion": {
    "options": {
      "messageStream": {
        "options": {
          "inputStream": {},
          "decoder": {
            "headerMarshaller": {},
            "messageBuffer": [],
            "isEndOfStream": false
          }
        }
      }
    }
  }
}

Expected Behavior

I would expect response to look like:

{
  "$metadata": {
    "httpStatusCode": 200,
    "requestId": "ae975225-b133-42c4-a8eb-0d425e650a40",
    "attempts": 1,
    "totalRetryDelay": 0
  },
  "contentType": "application/json",
  "sessionId": "96c1843f-8f91-4ff8-8742-753c4a0dbf91",
  "completion": {
    "chunk" : [
...chunk data here...
    ],
    "options": {
      "messageStream": {
        "options": {
          "inputStream": {},
          "decoder": {
            "headerMarshaller": {},
            "messageBuffer": [],
            "isEndOfStream": false
          }
        }
      }
    }
  }
}

Possible Solution

No response

Additional Information/Context

No response

zshzbh commented 2 weeks ago

Hey @davidgamify ,

Sorry to hear that you have this issue. I confirm that I can reproduce this issue. By looking at the InvokeAgentCommand Output definition, the type for the completion field is an asyncIterator object. Async iterators are similar to iterators but in this case the iteration is done over asynchronous values. You can also check the type of this field in the IDE.

In python boto3 pkg, the completion is EventStream. I think this is due to the data type difference among different languages. By checking the service doc - response syntax, if the action is successful, the service sends back an HTTP 200 response. there's no specific data type of the completion, since we use codeGen to convert service source code into different language sdk code, I'd say this would happen.

So, one way we could process the data retrieved is by using a for await loop, just like what you provided. please let me know if you have any other questions! :)

Thanks! Maggie

zshzbh commented 2 weeks ago

Posting my reproduction code for


import {
  BedrockAgentRuntimeClient,
  InvokeAgentCommand,
} from "@aws-sdk/client-bedrock-agent-runtime";

/**
 * @typedef {Object} ResponseBody
 * @property {string} completion
 */

/**
 * Invokes a Bedrock agent to run an inference using the input
 * provided in the request body.
 *
 * @param {string} prompt - The prompt that you want the Agent to complete.
 * @param {string} sessionId - An arbitrary identifier for the session.
 */
export const invokeBedrockAgent = async (prompt, sessionId) => {
  const client = new BedrockAgentRuntimeClient({ region: "us-east-1" });

  const agentId = "XXXX";
  const agentAliasId = "XXXX";

  const command = new InvokeAgentCommand({
    agentId,
    agentAliasId,
    sessionId: "123",
    inputText: "prompt",
  });

  try {
    let completion = "";
    const response = await client.send(command);

    console.log("res",response)
    console.log("response",response.completion)
    if (response.completion === undefined) {
      throw new Error("Completion is undefined");
    }

    console.log(response.completion);
    for await (let chunkEvent of response.completion) {
      const chunk = chunkEvent.chunk;
      console.log(chunk);
      const decodedResponse = new TextDecoder("utf-8").decode(chunk.bytes);
      completion += decodedResponse;
    }

    return { sessionId: sessionId, completion };
  } catch (err) {
    console.error(err);
  }
};

// Call function if run directly
import { fileURLToPath } from "url";
if (process.argv[1] === fileURLToPath(import.meta.url)) {
  const result = await invokeBedrockAgent("I need help.", "123");
  console.log(result);
}

Result :

res {
  '$metadata': {
    httpStatusCode: 200,
    requestId: '908d37bb-e5d4-4718-a74c-57507fd12f6e',
    extendedRequestId: undefined,
    cfId: undefined,
    attempts: 1,
    totalRetryDelay: 0
  },
  contentType: 'application/json',
  sessionId: '123',
  completion: SmithyMessageDecoderStream {
    options: {
      messageStream: [MessageDecoderStream],
      deserializer: [AsyncFunction (anonymous)]
    }
  }
}
response SmithyMessageDecoderStream {
  options: {
    messageStream: MessageDecoderStream { options: [Object] },
    deserializer: [AsyncFunction (anonymous)]
  }
}
SmithyMessageDecoderStream {
  options: {
    messageStream: MessageDecoderStream { options: [Object] },
    deserializer: [AsyncFunction (anonymous)]
  }
}
{
  bytes: Uint8Array(37) [
     83, 111, 114, 114, 121,  44,  32,  73,
     32,  99,  97, 110, 110, 111, 116,  32,
     97, 110, 115, 119, 101, 114,  32, 116,
    104,  97, 116,  32, 113, 117, 101, 115,
    116, 105, 111, 110,  46
  ]
}
{
  sessionId: '123',
  completion: 'Sorry, I cannot answer that question.'
}

davidgamify commented 2 weeks ago

Thanks @zshzbh - I am completely blocked by this issue. Hopefully, someone can come up with a resolution soon. I see it's a p2, so hopefully, it'll get attention soon.

zshzbh commented 2 weeks ago

Hey @davidgamify ,

Could you please help me understand why this is a blocker for you as you can get chunk by the for awaitloop?

davidgamify commented 2 weeks ago

@zshzbh - Because I don't get the chunk in the for await loop. I never get the chunk. Here's the output I get:

LOG  [DEBUG] 04:46.174 GamifyLog - !!!response:  {"$metadata":{"httpStatusCode":200,"requestId":"818a06e8-53db-43f9-9d48-9d08817f802e","attempts":1,"totalRetryDelay":0},"contentType":"application/json","sessionId":"e633d73f-e6ae-4851-b8e5-2b65e1a172f9","completion":{"options":{"messageStream":{"options":{"inputStream":{},"decoder":{"headerMarshaller":{},"messageBuffer":[],"isEndOfStream":false}}}}}}
[GoGamify] 'Bedrock error: ', [TypeError: source[Symbol.asyncIterator] is not a
function (it is undefined)]
 LOG  [DEBUG] 04:46.175 GamifyLog - !!!completion:  {
  "options": {
    "messageStream": {
      "options": {
        "inputStream": {},
        "decoder": {
          "headerMarshaller": {},
          "messageBuffer": [],
          "isEndOfStream": false
        }
      }
    }
  }
}
 ERROR  Bedrock error:  [TypeError: source[Symbol.asyncIterator] is not a function (it is undefined)]

zshzbh commented 2 weeks ago

Hey @davidgamify ,

I just realized that you are using react native. In this case, you have to install polyfills (ref) -

If you are consuming modular AWS SDK for JavaScript on react-native environments, you will need to add and import following polyfills in your react-native application:

react-native-get-random-values react-native-url-polyfill web-streams-polyfill

import "react-native-get-random-values";
import "react-native-url-polyfill/auto";
import "web-streams-polyfill/dist/polyfill";

import { DynamoDB } from "@aws-sdk/client-dynamodb";

Specifically Metro bundler used by react-native, enable Package Exports Support:

https://metrobundler.dev/docs/package-exports/ https://reactnative.dev/blog/2023/06/21/package-exports-support

Could you please confirm you have installed the required polyfills? If not, could you please install those and rebuild the app and try again?

Thanks! Maggie

davidgamify commented 2 weeks ago

I do have the polyfills and the get-random-values. I couldn't enable Package Exports Support as, when I did, axios wouldn't work. I'm trying to figure out the axios issue now.

davidgamify commented 2 weeks ago

I couldn't figure out the Axios issue. Unfortunately, I can't seem to get Axios to work when I enable Package Exports Support.

zshzbh commented 2 weeks ago

Is there anyway that you can use other artifacts instead of axios? What version of axios you are using? ASK SDK might have minimum version support for other pkgs. Could you please try to upgrade Axios to the latest version and try again?

If the issue persists, could you please send us the code with Axios so I can try to reproduce and troubleshoot on my end.

Thanks~! Maggie

aws / aws-sdk-js-v3