ChatBaiduWenxin stream mode doesn't work as expected

[Robert-ZLF]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain.js documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain.js rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

const model = new ChatBaiduWenxin({
    model: "ERNIE-Bot", // Available models are shown above
    streaming: true,
    temperature: 1,
    baiduApiKey: BAIDU_API_KEY, // In Node.js defaults to process.env.BAIDU_API_KEY
    baiduSecretKey: BAIDU_SECRET_KEY, // In Node.js defaults to process.env.BAIDU_SECRET_KEY
});

const stream = await model.stream("Hello! Tell me about yourself.");
const chunks = [];
for await (const chunk of stream) {
   chunks.push(chunk);
   console.log(`${chunk.content}|`);
}

Error Message and Stack Trace (if applicable)

Current output is return the entire final results

it should be streaming mode return like this

|
Hello|
!|
 I'm|
 an|
 AI|
 language|
 model|
 created|
 by|
 Open|
...

Description

When using ChatBaiduWenxin, how to return in a streaming fashion instead of returning a whole readable stream.

System Info

langchain 0.2.2 mac

[dosubot[bot]]

Hey @Robert-ZLF! I'm here to assist you with any issues or questions you have regarding LangChain.js. I can help you solve bugs, answer questions, and even guide you on how to become a contributor. Let's take a look at the issue you've posted and see how we can resolve it.

To get the ChatBaiduWenxin to return the response in a streaming fashion, you need to set the streaming property to true in the configuration object and provide a callback function to handle the new tokens as they are streamed. Here is an example configuration:

const config = {
  streaming: true,
  callbacks: [
    {
      async handleLLMNewToken(token: string) {
        // Handle each new token here
        console.log(token);
      },
    },
  ],
};

const chat = new ChatBaiduWenxin({
  modelName: "ERNIE-Bot",
  ...config,
});

const messages = [new HumanMessage("Hello!")];

const res = await chat.invoke(messages);

In this example, the handleLLMNewToken callback function will be called with each new token as it is generated, allowing you to process the response in a streaming fashion.

Additionally, you can use the .stream() method to handle streaming responses. This method returns a readable stream that you can iterate over. Here is an example:

const stream = chat.stream(messages);

for await (const chunk of stream) {
  console.log(chunk);
}

For models that do not support streaming, the entire response will be returned as a single chunk. You can also pipe a chat model into a StringOutputParser to extract just the raw string values from each chunk or use an HttpResponseOutputParser to stream bytes directly, such as for returning a stream in an HTTP response ^[1][2][3].

#### Details

**About Dosu** This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.

[jeasonnow]

ChatBaiduWenxin does not implement ChatBaiduWenxin.stream; it will default to calling the parent class and return data in non-streaming mode. If you want to use the stream mode, you can try this:

let result = ''
const model = new ChatBaiduWenxin({
    model: "ERNIE-Bot", // Available models are shown above
    streaming: true,
    temperature: 1,
    baiduApiKey: BAIDU_API_KEY, // In Node.js defaults to process.env.BAIDU_API_KEY
    baiduSecretKey: BAIDU_SECRET_KEY, // In Node.js defaults to process.env.BAIDU_SECRET_KEY,
    callbacks: [
      {
        async handleLLMNewToken(token: string) {
          result += token;
          console.log(`${token}|`);
        },
      },
    ],
});

@Robert-ZLF