[BUG] Pydantic model schema not added to system prompt

[olafgeibig]

Description

While investigating my problems how to make LiteLLM use OpenAI compatible APIs with structured output as reported in Issue 1333, I discovered that it started to work when I added the pydantic schema of the model to my prompt although I though that is is already happening under the hood of crewAI. Now I looked at my tracing (Arise Phoenix) and I saw that it is not doing that in the system prompt as it should be.

Pydantic out works, but only if I add the schema to the user prompt. As you can see in my tracing, the template is in the system prompt but not the schema. I'm afraid like that, the agent will loose its job 😂

Steps to Reproduce

1. Run my code using a model capable of structured output via openrouter, eg. Qwen 2.5
2. Check the system prompt in the tracing (note that the system prompt misses the schema)
3. Check the output (Model output did not adhere to the schema and the pydantic output is empty)

Expected behavior

Schema in the system prompt, model adheres to the schema and pydantic output has content.

Screenshots/Code snippets

load_dotenv()

writer_agent = Agent(
    role='Note writer',
    goal='Write personal notes using a specified template.',
    backstory='I am a note writer with a great skill to write personal notes.',
    verbose=True,
    llm="openrouter/qwen/qwen-2.5-72b-instruct",
    allow_delegation=False,
    tools=[]
)

note_task = Task(
    description=f"""
    Create a note about the resource Pydantic using the note template.
    The note shall contain a general section explaining the resource and its concepts and a multiple themed section with important links to websites.
    """,
    agent=writer_agent,
    expected_output="a note that is the filled template",
    output_pydantic=Note,
)    

crew = Crew(
    agents=[writer_agent],
    tasks=[note_task],
    verbose=True
)

crew_output = crew.kickoff()
print(f"Crew: {crew_output}")
print(f"Pydantic: {note_task.output.pydantic}")

Operating System

macOS Sonoma

Python Version

3.12

crewAI Version

0.61.0

crewAI Tools Version

n/a

Virtual Environment

Venv

Evidence

System prompt as logged by my tracing

"messages": [
    {
      "role": "system",
      "content": "You are Note writer. I am a note writer with a great skill to write personal notes.\nYour personal goal is: Write personal notes using a specified template.\nTo give my best complete final answer to the task use the exact following format:\n\nThought: I now can give a great answer\nFinal Answer: Your final answer must be the great and the most complete as possible, it must be outcome described.\n\nI MUST use these formats, my job depends on it!"
    },

Possible Solution

probably a bug when using non OpenAI models?

Additional context

Add the schema to the user prompt and the pydantic output works:

note_task = Task(
    description=f"""
    Create a note about the resource Pydantic using the note template.
    The note shall contain a general section explaining the resource and its concepts and a multiple themed section with important links to websites.
    Adhere to this schema: {Note.model_json_schema()}
    """,
    agent=writer_agent,
    expected_output="a note that is the filled template",
    output_pydantic=Note,
    output_file="note.json"
) 

[github-actions[bot]]

This issue is stale because it has been open for 30 days with no activity. Remove stale label or comment or this will be closed in 5 days.

[github-actions[bot]]

This issue was closed because it has been stalled for 5 days with no activity.