langchain-ai / langgraph

Build resilient language agents as graphs.
https://langchain-ai.github.io/langgraph/
MIT License
6.22k stars 983 forks source link

No useful traceback when using pydantic schema and failing validation #1978

Open eyurtsev opened 2 weeks ago

eyurtsev commented 2 weeks ago

Privileged issue

Issue Content

The code below fails on entry to ok_node, but the stack trace has no information about which node failed.

Given that the same schema is often shared between all nodes in the graph (at least by default) -- it makes it impossible to determine where the run time error is occurring.

from langgraph.graph import StateGraph, START, END
from typing_extensions import TypedDict

from pydantic import BaseModel

# The overall state of the graph (this is the public state shared across nodes)
class OverallState(BaseModel):
    a: str

def bad_node(state: OverallState):
    return {
        "a": 123 # Invalid
    }

def ok_node(state: OverallState):
    return {
        "a": "goodbye"
    }

# Build the state graph
builder = StateGraph(OverallState)
builder.add_node(bad_node)
builder.add_node(ok_node)
builder.add_edge(START, "bad_node")
builder.add_edge("bad_node", "ok_node")
builder.add_edge("ok_node", END)
graph = builder.compile()

# Test the graph with a valid input
graph.invoke({ "a": "hello"})

Part of error trace

354 return tasks

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/pregel/algo.py:495, in prepare_single_task(task_path, task_id_checksum, checkpoint, processes, channels, managed, config, step, for_execution, store, checkpointer, manager) 485 if triggers := sorted( 486 chan 487 for chan in proc.triggers (...) 492 > seen.get(chan, null_version) 493 ): 494 try: --> 495 val = next( 496 _proc_input( 497 step, proc, managed, channels, for_execution=for_execution 498 ) 499 ) 500 except StopIteration: 501 return

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/pregel/algo.py:627, in _proc_input(step, proc, managed, channels, for_execution) 625 # If the process has a mapper, apply it to the value 626 if for_execution and proc.mapper is not None: --> 627 val = proc.mapper(val) 629 yield val

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/graph/state.py:704, in _coerce_state(schema, input) 703 def _coerce_state(schema: Type[Any], input: dict[str, Any]) -> dict[str, Any]: --> 704 return schema(**input)

File ~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/pydantic/main.py:212, in BaseModel.init(self, **data) 210 # __tracebackhide__ tells pytest and some other tools to omit this function from tracebacks 211 tracebackhide = True --> 212 validated_self = self.pydantic_validator__.validate_python(data, self_instance=self) 213 if self is not validated_self: 214 warnings.warn( 215 'A custom validator is returning a value other than self.\n' 216 "Returning anything other than self from a top level model validator isn't supported when validating via `init__.\n" 217 'See themodel_validator` docs (https://docs.pydantic.dev/latest/concepts/validators/#model-validators) for more details.', 218 category=None, 219 )

ValidationError: 1 validation error for OverallState a Input should be a valid string [type=string_type, input_value=123, input_type=int] For further information visit https://errors.pydantic.dev/2.9/v/string_type

FrancisGajitos commented 1 day ago

Hey @eyurtsev, I'm in a group of 5 students from the University of Toronto and we're interested in looking into this issue! Just wanted to ask if there's any other important information that could help us fix this error traceback?

Thanks!

eyurtsev commented 1 day ago

I'd follow the stack trace to identify the location where the exception is raised.

File [~/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/graph/state.py:704](http://localhost:8888/home/eugene/.pyenv/versions/3.11.4/envs/core/lib/python3.11/site-packages/langgraph/graph/state.py#line=703), in _coerce_state(schema, input)
703 def _coerce_state(schema: Type[Any], input: dict[str, Any]) -> dict[str, Any]:
--> 704 return schema(**input)

And here figure out whether it's possible to catch the error and re-raise with a better exception or error message that has information about the current execution step or the previous step that caused a bad state update.

This could be a 1 line change or more (i haven't looked at what information is available).