Anthropic `tool_choice` should be `any`

josepablog commented 1 month ago

[x] This is actually a bug report.
[ ] I am not getting good LLM Results
[ ] I have tried asking for help in the community on discord or discussions and have not received a response.
[ ] I have tried searching the documentation and have not found an answer.

What Model are you using?

[ ] gpt-3.5-turbo
[ ] gpt-4-turbo
[ ] gpt-4
[x] Bedrock Anthropic Claude Sonnet 3.5

Describe the bug

Doesn't seem the tools are being sent to (Bedrock) Anthropic correctly. For instance, if I set set_verbose=True on the README example, I can see that tool_choice is not being sent:

{'model': 
     'claude-3-opus-20240229',
      'messages': [{'role': 'user', 'content': [{'type': 'text', 'text': 'Extract Jason is 25 years old.'}]}],
       'max_tokens': 1024,
       'tools': [{'name': 'User', 'description': 'Correctly extracted `User` with all the required parameters with correct types', 'input_schema': {'properties': {'name': {'title': 'Name', 'type': 'string'}, 'age': {'title': 'Age', 'type': 'integer'}}, 'required': ['age', 'name'], 'type': 'object'}}]}'

Per Anthropic's [documentation](https://docs.anthropic.com/en/docs/build-with-claude/tool-use] the default is:

auto: allows Claude to decide whether to call any provided tools or not. This is the default value.

Without this, in more complicated examples, Claude is not obliged to generate a valid JSON.

jxnl commented 1 month ago

would love a pr to fix this

josepablog commented 1 month ago

Would you mind pointing me to some files? I can certainly try if I get a chance

On Thu, Oct 17, 2024 at 6:29 PM Jason Liu @.***> wrote:

would love a pr to fix this

— Reply to this email directly, view it on GitHub https://github.com/instructor-ai/instructor/issues/1085#issuecomment-2421039076, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAN7KAI4L23K7XKAQBZJN3TZ4BP7LAVCNFSM6AAAAABQC3MZ32VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMRRGAZTSMBXGY . You are receiving this because you authored the thread.Message ID: @.***>

instructor-ai / instructor

Anthropic `tool_choice` should be `any` #1085