langchain-ai / langchain-google

MIT License
74 stars 78 forks source link

fix: ChatAnthropicVertex returning None during structured output #332

Closed kiarina closed 3 days ago

kiarina commented 4 days ago

When using with_structured_output with ChatAnthropicVertex, the server response is ignored, resulting in None being returned from the invoke method.

import os
from langchain_google_vertexai.model_garden import ChatAnthropicVertex
from langchain_core.pydantic_v1 import BaseModel
from langchain_core.messages import HumanMessage

model = ChatAnthropicVertex(
    project=os.environ["PROJECT_ID"],
    location="us-east5",
    model="claude-3-opus@20240229"
)

class MyModel(BaseModel):
    name: str
    age: int

model_with_structured_output = model.with_structured_output(MyModel)
message = HumanMessage(content="My name is Erick and I am 27 years old")
print(model_with_structured_output.invoke([message]))
# None

The server response at this time is as follows:

Message(id='msg_vrtx_01RuVwJ9ANqEjLKB2CciEz3r', content=[ToolUseBlock(id='toolu_vrtx_0147LC8SasuVZXi4RQsk1eow', input={'name': 'Erick', 'age': 27}, name='MyModel', type='tool_use')], model='claude-3-opus-20240229', role='assistant', stop_reason='tool_use', stop_sequence=None, type='message', usage=Usage(input_tokens=372, output_tokens=56))

In _format_output, the content of AIMessage becomes empty.

content=[] tool_calls=[{'name': 'MyModel', 'args': {'name': 'Erick', 'age': 27}, 'id': 'toolu_vrtx_0147LC8SasuVZXi4RQsk1eow'}]

Therefore, the condition in the following part of ToolsOutputParser does not retrieve data.

By making use of AIMessage.tool_calls when it is not empty, the issue is fixed.

After the fix, data can be retrieved as follows:

name='Erick' age=27
lkuligin commented 4 days ago

Huge thanks for the fix!

lkuligin commented 3 days ago

@kiarina linter is still failing, could you take a look, please?

kiarina commented 3 days ago

Sorry. I had a different version of ruff, so I modified some unnecessary files. After poetry install --with lint, I checked the modified files with make lint and undid the unnecessary modifications. Locally, all checks passed.

lkuligin commented 3 days ago

@kiarina please, feel free to send a PR to the contribution guide if you find that some things were unclear to a new contributor!