RobertCraigie
commented
2 months ago @kwnath thanks for the report and the detailed example notebook.
Can you share the python version you're using and the dependency versions you're using? e.g. with pip list.
I can't reproduce the issue you're seeing, with this script
from anthropic import Anthropic
client = Anthropic()
stream = client.messages.create(
model="claude-3-5-sonnet-20240620",
max_tokens=2048,
temperature=0.0,
tools=[
{
"name": "get_weather",
"description": "Get the current weather in a given location",
"input_schema": {
"type": "object",
"properties": {
"location": {
"type": "string",
"description": "The city and state, e.g. San Francisco, CA",
}
},
"required": ["location"],
},
}
],
tool_choice={"type": "any"},
stream=True,
messages=[{"role": "user", "content": "What's the weather like in San Francisco?"}],
)
for event in stream:
print(event)I get back multiple events
RawMessageStartEvent(message=Message(id='msg_01177R2njzZanvg1kphN3Wvh', content=[], model='claude-3-5-sonnet-20240620', role='assistant', stop_reason=None, stop_sequence=None, type='message', usage=Usage(input_tokens=366, output_tokens=10)), type='message_start')
RawContentBlockStartEvent(content_block=ToolUseBlock(id='toolu_01SjuyZhYAert5cA4LNzNNKc', input={}, name='get_weather', type='tool_use'), index=0, type='content_block_start')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='{"', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='location": ', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='"San Fran', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='ci', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='sco, CA', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockDeltaEvent(delta=InputJSONDelta(partial_json='"}', type='input_json_delta'), index=0, type='content_block_delta')
RawContentBlockStopEvent(index=0, type='content_block_stop')
RawMessageDeltaEvent(delta=Delta(stop_reason='tool_use', stop_sequence=None), type='message_delta', usage=MessageDeltaUsage(output_tokens=41))
RawMessageStopEvent(type='message_stop')
kwnath
Description
When calling the Anthropic client with
streaming=Trueandtool_choice={"type": "any"/"tool" }the output returns a tool call but with empty args.This is problematic for a few other reasons beyond no args being returned. For example, quite a few packages rely on the
anthropic-sdk, one of which islangchain-anthropic(ref).Expected response
I would expect that the output includes the inputs/args required for the tool call when streaming.
Reproduction steps
I've added a notebook to highlight some things: https://gist.github.com/kwnath/f42737c023767d5effdcca20cb5bd0a6