search

BerriAI / litellm

Call all LLM APIs using the OpenAI format. Use Bedrock, Azure, OpenAI, Cohere, Anthropic, Ollama, Sagemaker, HuggingFace, Replicate (100+ LLMs)
https://docs.litellm.ai/docs/
Other
10.12k stars 1.13k forks source link

[Feature]: Support bedrock offician tool usage #4326

Closed luccazifood closed 1 week ago

luccazifood commented 1 week ago

The Feature

https://docs.aws.amazon.com/bedrock/latest/userguide/tool-use.html

Motivation, pitch

Instead of modifying the prompt, aws api supports direct tool usage. performance and token usage is better. would be nice to use that way in litellm

Twitter / LinkedIn details

No response

luccazifood commented 1 week ago

if i find time, i can try to contribute

krrishdholakia commented 1 week ago

we already support it @luccazifood https://github.com/BerriAI/litellm/blob/dfd5882c313be35b0c62c726df69b338e5951b1a/litellm/llms/bedrock_httpx.py#L1765

krrishdholakia commented 1 week ago

Where in docs should we have had this?

P.S.: If you're free for 10mins this/next week - would love to learn how you're using litellm

https://calendly.com/d/4mp-gd3-k5k/litellm-1-1-onboarding-chat