Bug: Pydantic model errors during tool serialization

[znicholasbrown]

The following fails due to errors serialization the playwright_agent function and passing it to the scraping_agent:

from controlflow import flow as CFFlow, Task as CFTask, Agent
from playwright import sync_api

def playwright_agent(url: str) -> sync_api.Page:
    """
    Uses Playwright to scrape the site at the given URL and returns the playwright page object.
    """
    with sync_api.sync_playwright() as p:
        browser = p.chromium.launch(headless=True)
        page = browser.new_page()
        page.goto(url)

        return page 

scraping_agent = Agent(name="WesleyScrapes", tools=[playwright_agent])

@CFFlow
def scrape_details(url: str) -> str:
    """
    Given a url, this task scrapes the site for details about the tool.
    """
    return CFTask(
        objective="Scrape the site for details about the tool, following links to get more information if necessary.",
        result_type=str,
        agents=[scraping_agent],
        context={"url": url},
    ).run()

if __name__ == "__main__":
  scrape_details("https://prefect.io")

with the following:

16:23:26.558 | ERROR   | Flow run 'zealous-baboon' - Finished in state Failed("Flow run encountered an exception: PydanticSerializationError: Error calling function `_serialize_tools`: PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'playwright.sync_api._generated.Page'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.\n\nIf you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.\n\nFor further information visit https://errors.pydantic.dev/2.7/u/schema-for-unknown-type")
Traceback (most recent call last):
  File "/Users/nicholas/.pyenv/versions/workflow-webscraper/lib/python3.12/site-packages/pydantic/type_adapter.py", line 217, in __init__
    validator = _getattr_no_parents(type, '__pydantic_validator__')
                ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/.pyenv/versions/workflow-webscraper/lib/python3.12/site-packages/pydantic/type_adapter.py", line 98, in _getattr_no_parents
    raise AttributeError(attribute)
AttributeError: __pydantic_validator__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/nicholas/projects/flows/workflow-webscraper/cf_bug.py", line 31, in <module>
    scrape_details("https://prefect.io")
  File "/Users/nicholas/projects/prefect/src/prefect/flows.py", line 1326, in __call__
    return run_flow(
           ^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/flow_engine.py", line 798, in run_flow
    return run_flow_sync(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/flow_engine.py", line 678, in run_flow_sync
    return engine.state if return_type == "state" else engine.result()
                                                       ^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/flow_engine.py", line 243, in result
    raise self._raised
  File "/Users/nicholas/projects/prefect/src/prefect/flow_engine.py", line 635, in run_context
    yield self
  File "/Users/nicholas/projects/prefect/src/prefect/flow_engine.py", line 676, in run_flow_sync
    engine.call_flow_fn()
  File "/Users/nicholas/projects/prefect/src/prefect/flow_engine.py", line 655, in call_flow_fn
    result = call_with_parameters(self.flow.fn, self.parameters)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/utilities/callables.py", line 208, in call_with_parameters
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/decorators.py", line 110, in wrapper
    result = fn(*args, **kwargs)
             ^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/flows/workflow-webscraper/cf_bug.py", line 28, in scrape_details
    ).run()
      ^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/tasks.py", line 845, in __call__
    return run_task(
           ^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 918, in run_task
    return run_task_sync(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 731, in run_task_sync
    return engine.state if return_type == "state" else engine.result()
                                                       ^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 343, in result
    raise self._raised
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 673, in run_context
    yield self
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 729, in run_task_sync
    engine.call_task_fn(txn)
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 702, in call_task_fn
    result = call_with_parameters(self.task.fn, parameters)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/utilities/callables.py", line 208, in call_with_parameters
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/tasks/task.py", line 360, in run
    controller.run()
  File "/Users/nicholas/projects/prefect/src/prefect/tasks.py", line 845, in __call__
    return run_task(
           ^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 918, in run_task
    return run_task_sync(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 731, in run_task_sync
    return engine.state if return_type == "state" else engine.result()
                                                       ^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 343, in result
    raise self._raised
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 673, in run_context
    yield self
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 729, in run_task_sync
    engine.call_task_fn(txn)
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 702, in call_task_fn
    result = call_with_parameters(self.task.fn, parameters)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/utilities/callables.py", line 208, in call_with_parameters
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/controllers/controller.py", line 340, in run
    new_messages = self.run_once()
                   ^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/tasks.py", line 845, in __call__
    return run_task(
           ^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 918, in run_task
    return run_task_sync(**kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 731, in run_task_sync
    return engine.state if return_type == "state" else engine.result()
                                                       ^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 343, in result
    raise self._raised
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 673, in run_context
    yield self
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 729, in run_task_sync
    engine.call_task_fn(txn)
  File "/Users/nicholas/projects/prefect/src/prefect/task_engine.py", line 702, in call_task_fn
    result = call_with_parameters(self.task.fn, parameters)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/prefect/src/prefect/utilities/callables.py", line 208, in call_with_parameters
    return fn(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/controllers/controller.py", line 270, in run_once
    payload = self._setup_run()
              ^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/controllers/controller.py", line 192, in _setup_run
    tools.extend(task.get_tools())
                 ^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/tasks/task.py", line 526, in get_tools
    tools.extend([self._create_fail_tool(), self._create_success_tool()])
                  ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/tasks/task.py", line 468, in _create_fail_tool
    return Tool.from_function(
           ^^^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/llm/tools.py", line 83, in from_function
    parameters = TypeAdapter(fn).json_schema()
                 ^^^^^^^^^^^^^^^
  File "/Users/nicholas/.pyenv/versions/workflow-webscraper/lib/python3.12/site-packages/pydantic/type_adapter.py", line 223, in __init__
    core_schema, type, module, str(type), 'TypeAdapter', core_config, config_wrapper.plugin_settings
                               ^^^^^^^^^
  File "/Users/nicholas/projects/control-flow/src/controlflow/tasks/task.py", line 175, in __repr__
    serialized = self.model_dump()
                 ^^^^^^^^^^^^^^^^^
  File "/Users/nicholas/.pyenv/versions/workflow-webscraper/lib/python3.12/site-packages/pydantic/main.py", line 347, in model_dump
    return self.__pydantic_serializer__.to_python(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
pydantic_core._pydantic_core.PydanticSerializationError: Error calling function `_serialize_tools`: PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'playwright.sync_api._generated.Page'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.7/u/schema-for-unknown-type

Prior to launch this worked without issue so it's likely some tool serialization logic has changed.

[jlowin]

Thanks @znicholasbrown -- the change is that we started using return annotations to add more information to the tool description, but since your return annotation isn't Pydantic-serializable, it errored when trying to introspect it.

I've modified the logic to just ignore incompatible return annotations. However, function arguments must be serializable.