Closed ianva closed 1 month ago
Hey @ianva! I'm here to help you with any bugs, questions, or contributions you have for LangChain.js. Let's tackle this issue together while we wait for a human maintainer to join in.
To use the newly launched Structured Outputs feature of OpenAI with the response_format
configuration in LangChain.js, you should use the withStructuredOutput
method. This method allows you to create a model that returns output matching a JSON Schema or a Zod schema. Here is an example of how you can set it up:
import { z } from "zod";
import { zodToJsonSchema } from "zod-to-json-schema";
import { ChatOpenAI } from "@langchain/openai";
import { ChatPromptTemplate } from "@langchain/core/prompts";
// Define your Zod schema
const TranslationSchema = z.object({
translatedText: z.string(),
entities: z.array(
z.object({
ner: z.string(),
label: z.string(),
translated: z.string().optional(),
})
),
});
// Create the structured output model
const model = new ChatOpenAI({ modelName: "gpt-4-turbo-preview", temperature: 0 });
const modelWithStructuredOutput = model.withStructuredOutput({
schema: zodToJsonSchema(TranslationSchema),
name: "translation",
});
const prompt = ChatPromptTemplate.fromMessages([
"system",
`You are a helpful assistant. Translate the following text and extract named entities.`,
"human",
"Translate this text: {textToTranslate}",
]);
const chain = prompt.pipe(modelWithStructuredOutput);
const result = await chain.invoke({
textToTranslate: "Hello, world!",
});
console.log(result);
In this example:
TranslationSchema
) that specifies the structure of the expected output.withStructuredOutput
, passing in the Zod schema and additional options.This setup ensures that the output conforms to the specified JSON schema or Zod schema, leveraging the jsonMode
method to validate and parse the output accordingly. This approach is more reliable for generating structured results, even with higher temperature
values, making it suitable for more creative applications.
Note: If using jsonMode
as the method
, you must include context in your prompt about the structured output you want. This must include the keyword: JSON
[1][2][3][4][5].
To continue talking to Dosu, mention @dosu.
I bet the answer is not correct. It should be smth like
.withStructuredOutput(TranslationSchema, { name: "translate", });
And again with latest 3.1 update and tooling it seems to me functionality is broken somehow. I cant make model to generate appropriate content
Hey, we're implementing json_schema
response format in #6438
Is the encapsulation of the ChatOpenAI class too complex, making it difficult to expand fields? Could there be some configurations that allow for merging when new fields are added?
Updating the response_format
type to allow
{
type: "json_schema",
json_schema: { ... }
}
is simple, however OpenAI added a new beta endpoint for making requests with this new json schema response format, as well as some utils for converting Zod schemas to their expected JSON schema format. Due to this, it's not as simple as expending the type, and actual code logic changes are required to make it work as intended.
Thanks for your work, looking forward to this update arriving soon
I am curious shouldn't withStructureOuput
directly implement this whe calling OpenAI? cc/ @bracesproul
@david8z We can't make withStructuredOutput
call with json_schema
mode by default as that would technically be a breaking change (any change to default behavior = "breaking" change). I believe we are planning on making withStructuredOutput
pass strict: true
by default when we bump minor versions next.
Checked other resources
Example Code