guardrails-ai / guardrails

Adding guardrails to large language models.
https://www.guardrailsai.com/docs
Apache License 2.0
3.6k stars 259 forks source link

Adds support for json structured return through tools #840

Closed dtam closed 1 week ago

dtam commented 1 week ago

adds support for tools based function calling as a strategy for from_pydantic guards to achieve structured output see usage below. this is only supported on openAI models that support function calling and

class Task(BaseModel):
    status: str
    priority: int
    description: str

guard = Guard.from_pydantic(Tasks)
tools = [] # openai function calling tool array
output = guard(
      openai.chat.completions.create,
      model='gpt-4o',
      msg_history=[
          {
              "role": "user",
              "content": "You are a helpful assistant"
              "read this email and return the tasks from it."
              " some email blah blah blah.",
          }
      ],
      tools=guard.add_json_function_calling_tool(tools),
      tool_choice="required",
)

note List[BaseModel] as a base_model is not supported as its technically not a pydantic object. It can be achieved with two models see example below

    class Task(BaseModel):
        status: str
        priority: int
        description: str

    class Tasks(BaseModel):
        list: List[Task]

    guard = Guard.from_pydantic(Tasks)

fixes #846

CalebCourier commented 1 week ago

Question about this:

note List[BaseModel] as a base_model is not supported as its technically not a pydantic object

How does this lack of support manifest? We were previously supporting function calling for root level lists here: https://github.com/guardrails-ai/guardrails/blob/main/guardrails/utils/pydantic_utils/v2.py#L130

The output_schema should also support top level lists.

[Edit]: I'm guessing this is what you're talking about: https://github.com/guardrails-ai/guardrails/pull/835/files#r1646518102

dtam commented 1 week ago

Question about this:

note List[BaseModel] as a base_model is not supported as its technically not a pydantic object

How does this lack of support manifest? We were previously supporting function calling for root level lists here: https://github.com/guardrails-ai/guardrails/blob/main/guardrails/utils/pydantic_utils/v2.py#L130

The output_schema should also support top level lists.

[Edit]: I'm guessing this is what you're talking about: https://github.com/guardrails-ai/guardrails/pull/835/files#r1646518102

neither the openai tools or functions api does supports array as the base type of the parameter tool generated by convert_pydantic_model_to_openai_fn( with List[Delivery] Error code: 400 - {'error': {'message': 'Invalid schema for function \'Delivery\': schema must be a JSON Schema of \'type: "object"\', got \'type: "array"\'.', 'type': 'invalid_request_error', 'param': 'functions[0].parameters', 'code': 'invalid_function_parameters'}} tool generated with my code with List[Delivery] Error code: 400 - {'error': {'message': 'Invalid schema for function \'gd_response_tool\': schema must be a JSON Schema of \'type: "object"\', got \'type: "array"\'.', 'type': 'invalid_request_error', 'param': 'tools[0].function.parameters', 'code': 'invalid_function_parameters'}}.