search

langchain-ai / langchain-aws

Build LangChain Applications on AWS
MIT License
103 stars 79 forks source link

Refactor ChatBedrock to improve support for providers #224

Open 3coins opened 3 weeks ago

3coins commented 3 weeks ago

Problem

Bedrock supports several providers (AI21, Anthropic, Amazon, Mistral, Meta etc.), each of which has a slightly different request/response structure. The current ChatBedrock class currently accommodates these providers by incorporating several conditional blocks which have become unsustainable as more providers, models and features have been introduced in Bedrock. This has also made it difficult and error prone to debug and add bug fixes over time.

Solution

Refactor the ChatBedrock class to individual provider classes, for example ChatBedrockAnthropic, and provide a mechanism in ChatBedrock to switch to these, so the users of don't have to create these classes separately.

Forma-Lin commented 1 week ago

Hello, I'm a student from University of Toronto. Me and my team from class are looking for ways we can contribute to langchain. Is this issue intended to solve the different promts model providers accept? How's the expected result different with ChatBedrockConverse class? It looks like also a solution to this issue. Thank you so much for your time reading this.

cab938 commented 1 week ago

I'm not @3coins but I'm happy to share my frustrations @Forma-Lin . For instance, I use llama models on bedrock, but right now the langchain_aws module doesn't include support for many (any?) of the features within the llama 3.1+ models with respect to tool calling or vision. It would be ideal imo to have the ability to use various models with bedrock and get model-specific support functions.

Forma-Lin commented 14 minutes ago

Hi @cab938, thank you so much for the answer. I totally get it now. Do you consider this class refactoring as a reasonable task for four fourth-year university students to complete in a month?