OpenAI AzureChatOpenAI doesn't support the new structured output capability even though BaseChatOpenAI does

mgberg commented 1 month ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

The following code doesn't work:

class Person(BaseModel):
    first_name: str
    last_name: str

LLM = AzureChatOpenAI(
    azure_deployment="gpt-4o-2024-08-06", api_version="2024-08-01-preview", temperature=0
).with_structured_output(schema=Person, method="json_schema", strict=True)
result = LLM.invoke(HumanMessage("Who was the first president of the United States?"))

Error Message and Stack Trace (if applicable)

The exception received was:

Received unsupported arguments {'strict': True}
  File "/home/goldbermg3/GitProjects/remis-pdf-utils/structured_out_test.py", line 13, in <module>
    ).with_structured_output(schema=Person, method="json_schema", strict=True)
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
ValueError: Received unsupported arguments {'strict': True}

Description

I expected the new Structured Output capability (which is available through Azure) which is implemented on BaseChatOpenAI to work using AzureChatOpenAI. However, after browsing the code, I saw that the support for mode="json_schema" and the strict argument is implemented only for BaseChatOpenAI; AzureChatOpenAI overrides BaseChatOpenAI.with_structured_outputs and does not include support for those arguments.

System Info

System Information

OS: Linux OS Version: #1 SMP Fri Mar 29 23:14:13 UTC 2024 Python Version: 3.11.9 (main, Apr 6 2024, 17:59:24) [GCC 11.4.0]

Package Information

langchain_core: 0.2.35 langchain: 0.2.15 langsmith: 0.1.104 langchain_openai: 0.1.23 langchain_text_splitters: 0.2.2

Optional packages not installed

langgraph langserve

Other Dependencies

aiohttp: 3.10.5 async-timeout: Installed. No version info available. httpx: 0.27.0 jsonpatch: 1.33 numpy: 1.26.4 openai: 1.42.0 orjson: 3.10.7 packaging: 24.1 pydantic: 2.8.2 PyYAML: 6.0.2 requests: 2.32.3 SQLAlchemy: 2.0.32 tenacity: 8.5.0 tiktoken: 0.7.0 typing-extensions: 4.12.2

arne-rustad commented 1 month ago

Is there any plan for this to be fixed?

PvanHengel commented 1 month ago

Whats the update here?

arne-rustad commented 4 weeks ago

How can we get this fixed? I would assume most businesses in Europe do not use the OpenAI API directly, but instead through Azure. Thus, this is a pretty bad limitation that it is not implemented for the Azure version of the API.

Emilyho11 commented 3 weeks ago

Hello, my group and I are from the University of Toronto, and we are going to take a look and work on this issue, hopefully to have a working PR soon.

arne-rustad commented 3 weeks ago

Hello, my group and I are from the University of Toronto, and we are going to take a look and work on this issue, hopefully to have a working PR soon.

Hi Emily, I really appreciate you volunteering to work on this. However, I believe the issue has been fixed already. I've seen no update about it, but I was checking the code and saw that it was updated. Tested it a couple of days ago and it appears to now be working. It only works for GPT-4o, not GPT-4o-mini, but that is an Azure issue, not a Langchain issue. Even though OpenAI released strict structured output for GPT-4o-mini two months ago, Azure is yet to do the same.

baskaryan commented 2 weeks ago

should be fixed as of #26599 which was released in langchain-openai 0.2.2. @arne-rustad @PvanHengel @mgberg could you check if that fixes?

langchain-ai / langchain