add with_structured_output support for Pydantic models, dicts and Enum

[mattf]

Bind a structured output schema to the model.

The schema can be -

0. a dictionary representing a JSON schema
1. a Pydantic object
2. an Enum

0. If a dictionary is provided, the model will return a dictionary. Example:

   
   json_schema = {
   "title": "joke",
   "description": "Joke to tell user.",
   "type": "object",
   "properties": {
       "setup": {
           "type": "string",
           "description": "The setup of the joke",
       },
       "punchline": {
           "type": "string",
           "description": "The punchline to the joke",
       },
   },
   "required": ["setup", "punchline"],
   }

structured_llm = llm.with_structured_output(json_schema) structured_llm.invoke("Tell me a joke about NVIDIA")

Output: {'setup': 'Why did NVIDIA go broke? The hardware ate all the software.',

'punchline': 'It took a big bite out of their main board.'}

1. If a Pydantic schema is provided, the model will return a Pydantic object.
    Example:

from langchain_core.pydantic_v1 import BaseModel, Field class Joke(BaseModel): setup: str = Field(description="The setup of the joke") punchline: str = Field(description="The punchline to the joke")

structured_llm = llm.with_structured_output(Joke) structured_llm.invoke("Tell me a joke about NVIDIA")

Output: Joke(setup='Why did NVIDIA go broke? The hardware ate all the software.',

punchline='It took a big bite out of their main board.')

2. If an Enum is provided, all values must be strings, and the model will return
    an Enum object. Example:

import enum class Choices(enum.Enum): A = "A" B = "B" C = "C"

structured_llm = llm.with_structured_output(Choices) structured_llm.invoke("What is the first letter in this list? [X, Y, Z, C]")

Output: <Choices.C: 'C'>

Note about streaming: Unlike other streaming responses, the streamed chunks
will be increasingly complete. They will not be deltas. The last chunk will
contain the complete response.

For instance with a dictionary schema, the chunks will be:

structured_llm = llm.with_structured_output(json_schema) for chunk in structured_llm.stream("Tell me a joke about NVIDIA"): print(chunk)

Output:

{}

{'setup': ''}

{'setup': 'Why'}

{'setup': 'Why did'}

{'setup': 'Why did N'}

{'setup': 'Why did NVID'}

...

{'setup': 'Why did NVIDIA go broke? The hardware ate all the software.', 'punchline': 'It took a big bite out of their main board'}

{'setup': 'Why did NVIDIA go broke? The hardware ate all the software.', 'punchline': 'It took a big bite out of their main board.'}

For instnace with a Pydantic schema, the chunks will be:

structured_llm = llm.with_structured_output(Joke) for chunk in structured_llm.stream("Tell me a joke about NVIDIA"): print(chunk)

Output:

setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline=''

setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It'

setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It took'

...

setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It took a big bite out of their main board'

setup='Why did NVIDIA go broke? The hardware ate all the software.' punchline='It took a big bite out of their main board.'

For Pydantic schema and Enum, the output will be None if the response is
insufficient to construct the object or otherwise invalid. For instance,

llm = ChatNVIDIA(max_tokens=1) structured_llm = llm.with_structured_output(Joke) print(structured_llm.invoke("Tell me a joke about NVIDIA"))

Output: None