How to combine String Output Parser and JSON Output Functions Parser into one stream pipline?

[arvinxx]

By using the default stream mode:

const langChainMessages = mapToLangChainMessages(messages);

const http = new HttpResponseOutputParser();

const stream = await model.pipe(http).stream(langChainMessages);

return new Response(stream);

it works well with default user content:

But when user try to call with OpenAI Function Call, the result is empty:

and I try to trace it in the LangSmith (trace link):

we can see that the ai is responsing correct with tool_calls. But the HttpResponseOutputParser is result with empty output.

And I have also try with implement like below:

const stream = await model
  .pipe(new StringOutputParser())
  .pipe(new JsonOutputFunctionsParser({ diff: true }))
  .stream(langChainMessages);

But it aslo not working. Here is the trace link: https://smith.langchain.com/public/971acb7b-1550-4377-aeb0-70e630d16c62/r

I think the HttpResponseOutputParser should support to parse both string content and function call, otherwise we can't use it in the real product.

[dosubot[bot]]

🤖

Hey @arvinxx! Good to see you back here again. I hope you're enjoying your journey with langchainjs as much as we are enjoying providing it! 😄

To integrate the StringOutputParser and JsonOutputFunctionsParser into a single stream pipeline, you can follow the example below:

import { StringOutputParser } from "langchain/output_parsers";

// ...

const chain = prompt
  .pipe(model)
  .pipe(new StringOutputParser())
  .pipe(new JsonOutputFunctionsParser({ diff: true }));

const stream = await chain.stream({
  foo: "bears",
});

for await (const chunk of stream) {
  console.log(chunk);
}

This code will first pass the output from the model through the StringOutputParser, which will convert the output into a string. Then it will pass that string through the JsonOutputFunctionsParser, which will convert the string into a JSON object. The resulting stream will emit JSON patch operations as the output is updated.

The StringOutputParser in the langchainjs codebase is a class that extends the BaseTransformOutputParser. Its main functionality is to parse the string output from a Language Learning Model (LLM) call. This is done in the parse method, which takes a string as an argument and returns a promise of the parsed output. In this case, the parsed output is the same as the input string, as indicated by return Promise.resolve(text);.

Unfortunately, I couldn't find specific information about the JsonOutputFunctionsParser in the langchainjs codebase. Could you provide more details about this class or its usage? This would help me give a more accurate answer.

Sources

• examples/src/prompts/json_structured_output_parser_streaming.ts
• langchain-core/src/output_parsers/string.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[arvinxx]

@dosubot no, it's not useful. and I think is a bug

[dosubot[bot]]

🤖

Based on the information you've provided, it seems like you're trying to combine the StringOutputParser and JsonOutputFunctionsParser into a single stream pipeline. However, you're encountering an issue where the HttpResponseOutputParser is returning an empty output when used with OpenAI Function Call.

To address this, you might want to consider using the CombiningOutputParser class. This class is designed to take multiple parsers as arguments and combine their results. Here's an example of how you can use it:

const answerParser = StructuredOutputParser.fromNamesAndDescriptions({
  answer: "answer to the user's question",
  source: "source used to answer the user's question, should be a website.",
});

const confidenceParser = new RegexParser(
  /Confidence: (A|B|C), Explanation: (.*)/,
  ["confidence", "explanation"],
  "noConfidence"
);

const parser = new CombiningOutputParser(answerParser, confidenceParser);

In this code, answerParser is a StructuredOutputParser that parses the JSON output, and confidenceParser is a RegexParser that parses the string output. The CombiningOutputParser named parser combines the results of these two parsers.

You can then use the CombiningOutputParser's parse method to parse the response from the OpenAI model:

console.log(await parser.parse(response));
/*
{
  answer: 'Paris',
  source: 'https://www.worldatlas.com/articles/what-is-the-capital-of-france.html',
  confidence: 'A',
  explanation: 'The capital of France is Paris.'
}
*/

This code logs the combined results of the StructuredOutputParser and RegexParser. You can find more details about this in the examples/src/prompts/combining_parser.ts file.

As for the HttpResponseOutputParser, it's a class that is used to parse the output of HTTP responses. It's used in conjunction with a language model to stream the output of the model's responses. You can find more details about this in the langchain/src/output_parsers/tests/http_response.test.ts file.

I hope this helps! If you have any other questions or need further clarification, feel free to ask.

Sources

• examples/src/prompts/combining_parser.ts
• langchain/src/output_parsers/tests/http_response.test.ts

---

This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

If you want to continue the conversation, start your reply with @dosu-bot.

[arvinxx]

@jacoblee93 For your inference, the ai SDK support to streaming the function call result with pure string: https://sdk.vercel.ai/docs/guides/providers/openai-functions#recursive-function-calls

the response：

[jacoblee93]

Hey! Does something like the last example on this page work for you?

You can pass an internal parser to the HttpResponseOutputParser.

https://js.langchain.com/docs/modules/model_io/output_parsers/types/http_response

Let me know if it doesn't and we can figure something out. Thanks for all the feedback so far!

[arvinxx]

@jacoblee93 I try with code below, it's not useful.

const http = new HttpResponseOutputParser({
  contentType: 'text/event-stream',
  outputParser: new JsonOutputFunctionsParser({ diff: true }),
});

const stream = await model.pipe(http).stream(langChainMessages);

return new Response(stream);

the default output is:

my scene need two abilities:

1. if assistant message is pure text output , response with text.
2. if assistant message is a function call result, also response it by pure text.

and the paser should combine the both strategies.

[jacoblee93]

I see - you could define your own custom output parser by extending the base class:

https://github.com/langchain-ai/langchainjs/blob/main/langchain-core/src/output_parsers/string.ts#L21 https://github.com/langchain-ai/langchainjs/blob/main/langchain-core/src/output_parsers/transform.ts#L19

but I can also see if we can make one that makes sense in the next few days!

[arvinxx]

I think an offical parser is a must and there also should be an example with it. Because default text response + function call response is becoming a more worldwide usage case when the community is exploring more potential of LLM other than only chat completion.

[arvinxx]

@jacoblee93 Hi, is this issue any update?

[jacoblee93]

Hey, not yet :( have been slammed but will try to get to it as soon as I can!

[arvinxx]

@jacoblee93 Thanks for your great work. I have been blocked by this for about a week...

Actually I have try to write a custom output parser but failed to run as my expect.

There is few document about how to write the custom output parser (I don't know the process of methods like parseResult / parse / parsePartialResult), and the official parsers seem not good examples to follow...

[limcolin]

Hey! Does something like the last example on this page work for you?

You can pass an internal parser to the HttpResponseOutputParser.

https://js.langchain.com/docs/modules/model_io/output_parsers/types/http_response

Let me know if it doesn't and we can figure something out. Thanks for all the feedback so far!

Can confirm that this method works - specifically:

Or pass a custom output parser to internally parse chunks for e.g. streaming function outputs:

import { ChatOpenAI } from "@langchain/openai";
import {
  HttpResponseOutputParser,
  JsonOutputFunctionsParser,
} from "langchain/output_parsers";

const handler = async () => {
  const parser = new HttpResponseOutputParser({
    contentType: "text/event-stream",
    outputParser: new JsonOutputFunctionsParser({ diff: true }),
  });

  const model = new ChatOpenAI({ temperature: 0 }).bind({
    functions: [
      {
        name: "get_current_weather",
        description: "Get the current weather in a given location",
        parameters: {
          type: "object",
          properties: {
            location: {
              type: "string",
              description: "The city and state, e.g. San Francisco, CA",
            },
            unit: { type: "string", enum: ["celsius", "fahrenheit"] },
          },
          required: ["location"],
        },
      },
    ],
    // You can set the `function_call` arg to force the model to use a function
    function_call: {
      name: "get_current_weather",
    },
  });

  const stream = await model.pipe(parser).stream("Hello there!");

  const httpResponse = new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
    },
  });

  return httpResponse;
};

await handler();

https://js.langchain.com/docs/modules/model_io/output_parsers/types/http_response

Been racking my brain trying to get this to work for months now.

Thanks for your work @jacoblee93