ChatVertexAI: with_structured_output with dict schema raises InvalidArgument error

[nndnha]

Sample code:

from langchain_core.pydantic_v1 import BaseModel
from langchain_core.utils.function_calling import convert_to_openai_tool
from langchain_google_vertexai import ChatVertexAI

class AnswerWithJustification(BaseModel):
    """An answer to the user question along with justification for the answer."""
    answer: str
    justification: str

dict_schema = convert_to_openai_tool(AnswerWithJustification)
llm = ChatVertexAI(model_name="gemini-pro", temperature=0)
structured_llm = llm.with_structured_output(dict_schema)

structured_llm.invoke("What weighs more a pound of bricks or a pound of feathers")

Output:

Retrying langchain_google_vertexai.chat_models._completion_with_retry.<locals>._completion_with_retry_inner in 4.0 seconds as it raised InvalidArgument: 400 Request contains an invalid argument..

Dependencies:

langchain==0.2.3
langchain-community==0.2.4
langchain-core==0.2.5
langchain-google-vertexai==1.0.5

[tdigangi]

It appears that the convert_to_openai_tool structure changed how the parameters were output. Also the tools attributes name and description have to be appended when using a dict schema as they are not automatically inferred.

I suggest using convert_to_openai_function in its place. I've updated documentation in code to better refect langchain-core https://github.com/langchain-ai/langchain-google/pull/311