BedrockRuntimeClient ConverseStreamCommand can't receive any response in react native

[zhu-xiaowei]

Checkboxes for prior research

• [X] I've gone through Developer Guide and API reference
• [X] I've checked AWS Forums and StackOverflow.
• [X] I've searched for previous similar issues and didn't find any solution.

Describe the bug

When using BedrockRuntimeClient in React Native environment, InvokeModelCommand works well, but ConverseStreamCommand and InvokeModelWithResponseStreamCommand will not return any response.

And I've read the getting-started for react native and added following imports

import "react-native-get-random-values";
import "react-native-url-polyfill/auto";
import "web-streams-polyfill/dist/polyfill";

then added blow code in metro.config.js

const config = {
  transformer: {
    getTransformOptions: async () => ({
      transform: {
        experimentalImportSupport: true, // false is still not work. 
        inlineRequires: false,
      },
    }),
  },
};

module.exports = mergeConfig(getDefaultConfig(__dirname), config);

the bedrock code as:

const command = new ConverseStreamCommand({
  messages: messages,
  modelId,
});
const apiResponse = await client.send(command);
console.log(JSON.stringify(apiResponse.stream));

for await (const item of apiResponse.stream) {
  console.log(item);
  if (item.contentBlockDelta) {
    const text = item.contentBlockDelta.delta?.text;
    console.log(text);
  }
}

client.send(command) works well, and the log for apiResponse.stream is:

{"options":{"messageStream":{"options":{"inputStream":{},"decoder":{"headerMarshaller":{},"messageBuffer":[],"isEndOfStream":false}}}}}

The for await block never goes inside, then I try to remove the transformer in metro.config.js and the apiResponse.stream will return the following error:

 WARN  Invalid responseType: blob
 ERROR [TypeError: Cannot read property 'blobId' of undefined]
 WARN  Invalid responseType: blob
 WARN  Invalid responseType: blob
 WARN  Invalid responseType: blob

I don't know what configuration I'm missing or is there any sample code for calling methods of ConverseStreamCommandor InvokeModelWithResponseStreamCommand in a React Native environment?

SDK version number

"@aws-sdk/client-bedrock-runtime": "^3.614.0"

Which JavaScript Runtime is this issue in?

React Native

Details of the browser/Node.js/ReactNative version

"react-native": "0.74.5"

Reproduction Steps

see the above describe.

Observed Behavior

ConverseStreamCommand command send successfully but apiResponse.stream receive without any streaming response.

Expected Behavior

can receive the streaming response for ConverseStreamCommand

Possible Solution

No response

Additional Information/Context

The credentials are configured correctly and the same code runs perfectly in the node environment.

[aBurmeseDev]

HI @zhu-xiaowei - thanks for reaching out.

It sounds like SDK call was made successfully as you receive the response from client but not from apiResponse.stream. I quickly attempted to reproduce it in Node env with simple code below and wasn't able to reproduce.

import {
  BedrockRuntimeClient,
  ConverseStreamCommand,
} from "@aws-sdk/client-bedrock-runtime"; // ES Modules import
const client = new BedrockRuntimeClient(config);
const input = {
  modelId: "STRING_VALUE",
  messages: messages,
};
const command = new ConverseStreamCommand(input);
const response = await client.send(command);

Before i attempt to reproduce with React Native, can you clarify a few things:

• Which Bedrock model are you using?
• Have you had a chance to try it in different env like Node or browser?
• Are you using Expo when setting up React Native? Can you share steps you took to set up your React Native env?

[zhu-xiaowei]

Hi @aBurmeseDev, thanks for working on this issue, here are more details:

1. I am using the anthropic.claude-3-sonnet-20240229-v1:0 model, it looks like this issue is encountered in all straming modes, not limited to a certain model in React Native env.
2. Node and browser work fine.
3. I am using a bare React Native environment created with react-native-cli, see the official documentation Getting Started Without a Framework, we are not using Expo.

[zhu-xiaowei]

To correct, the error above is because I also added another polyfill: import 'react-native-polyfill-globals/auto'; After removing this polyfill, error still appears, and the error content is:

[TypeError: Object is not async iterable]

Then I tried to add polyfill import '@azure/core-asynciterator-polyfill'; After adding this polyfill, the error becomes:

[TypeError: source[Symbol.asyncIterator] is not a function (it is undefined)]

In short, I can't find a way to successfully read the results returned by the straming model in the React Native environment:

const apiResponse = await client.send(command);

for await (const item of apiResponse.stream) {
  console.log(item); // can't get the streaming response.
}

[zhu-xiaowei]

@aBurmeseDev any updates?

[aBurmeseDev]

Hi @zhu-xiaowei - sorry for the delay. I haven't been able to reproduce it on my end. Could you please provide the complete code snippet, including the part where you make the API call using the SDK so that I can better understand and assist you with the issue you're facing?

[zhu-xiaowei]

Hi @aBurmeseDev, I created a demo app repo using @react-native-community/cli@latest and only modified the App.tsx file.

In App.tsx, I added the BedrockRuntimeClient and InvokeModelWithResponseStreamCommand to obtain a streaming response. The prompt is a simple "Hi".

The expected behavior is for the response to be displayed in the streaming resultText.

You can add your accessKeyId and secretAccessKey for testing purposes.

The current issue is that new BedrockRuntimeClient(...) is not functioning properly. A console warning appears, stating:

Therefore, I am currently unable to run the InvokeModelWithResponseStreamCommand and thus cannot reproduce the problem mentioned above. I would like to know if new BedrockRuntimeClient(...) is working correctly on your end.

[aBurmeseDev]

@zhu-xiaowei - thanks for providing further details and repo. I was able to reproduce it and we've identified the bug The fix was released in v3.675.0. Please verify that it's working on your end without warning.

If issue persists after upgrading to version 3.675.0, please remove the node_modules and/or the package-lock file, and then reinstall the required packages. This step can often resolve lingering issues related to dependencies and package versions.

Hope that resolves the issue! Best, John

[zhu-xiaowei]

Hi @aBurmeseDev, after I upgrade to version v3.675.0 the above issue was resolved, but another issue appeared:

So invokeModelWithResponseStream still cannot execute and provide streaming response. And I added import "web-streams-polyfill/dist/polyfill"; at the top of the file, you can reproduce this issue in the demo app repo.

[zhu-xiaowei]

Any updates?