Bedrock not working with Claude 3

jarib commented 7 months ago

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain.js documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain.js rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import { BedrockChat } from '@langchain/community/chat_models/bedrock'

async function main() {
    try {
        const model = new BedrockChat({
            model: 'anthropic.claude-3-sonnet-20240229-v1:0'
        });

        const result = await model.invoke('Tell me a story');
        console.log(result);
    } catch (error) {
        console.error(error);
        process.exit(1);
    }
}

main()

Error Message and Stack Trace (if applicable)

Error: Error 400: "claude-3-sonnet-20240229" is not supported on this API. Please use the Messages API instead.
    at BedrockChat._call (file:///Users/user/src/langchain-repro/node_modules/@langchain/community/dist/chat_models/bedrock/web.js:230:19)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async BedrockChat._generate (file:///Users/user/src/langchain-repro/node_modules/@langchain/core/dist/language_models/chat_models.js:347:22)
    at async Promise.allSettled (index 0)
    at async BedrockChat._generateUncached (file:///Users/user/src/langchain-repro/node_modules/@langchain/core/dist/language_models/chat_models.js:110:25)
    at async BedrockChat.invoke (file:///Users/user/src/langchain-repro/node_modules/@langchain/core/dist/language_models/chat_models.js:50:24)
    at async main (file:///Users/user/src/langchain-repro/test.js:9:24)

Description

These two issues from Python also apply to the JS version:

https://github.com/langchain-ai/langchain/issues/18513 https://github.com/langchain-ai/langchain/issues/18514

System Info

Platform: Mac Node: v20.11.0 Yarn version: 1.22.19

dosubot[bot] commented 7 months ago

Based on the error message you're receiving, it seems that the model identifier you're using, "claude-3-sonnet-20240229", is not supported by the API you're trying to use. This is likely the root cause of the issue you're experiencing.

The BedrockChat class in LangChain.js is designed to interact with specific language models. When you instantiate a new BedrockChat object, you need to provide a model identifier that is supported by the API. In your case, the model identifier "claude-3-sonnet-20240229" seems to be unsupported, hence the error.

To resolve this issue, you should replace the unsupported model identifier with a supported one. Unfortunately, without the exact list of supported model identifiers, I cannot provide a direct code modification to resolve this issue. You might need to check the official documentation or contact support for LangChain.js or the Bedrock model to find the correct model identifier for Claude 3.

Here's a general example of how you might modify your code once you have the correct model identifier:

import { BedrockChat } from '@langchain/community/chat_models/bedrock'

async function main() {
    try {
        const model = new BedrockChat({
            model: 'supported.model.identifier' // Replace with the correct model identifier
        });

        const result = await model.invoke('Tell me a story');
        console.log(result);
    } catch (error) {
        console.error(error);
        process.exit(1);
    }
}

main()

Regarding the two Python issues you mentioned, it's possible that they are related to the issue you're experiencing. However, without more information, it's hard to say for sure. If the issues are indeed related, the solutions proposed in those issues might also apply to your case. However, since the issues are in Python and you're working with JavaScript, the solutions might not be directly applicable.

I hope this helps! If you have any more questions or need further clarification, feel free to ask.

Sources

#### About Dosu

This response is meant to be useful and save you time. It is **not meant to be a precise solution**, but rather a starting point for your own research. Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant. If you want to continue the conversation, start your reply with @dosu-bot.

jacoblee93 commented 7 months ago

Will patch this tonight! Thanks for flagging.

jacoblee93 commented 7 months ago

Hey so this is actually going to take me a bit longer as I don't have access to the model on Bedrock :(

Will aim to get it as soon as I can and then work on this.

jarib commented 7 months ago

It's definitely already available:

$ aws bedrock list-foundation-models --region 'us-east-1' | jq '.modelSummaries[].modelId' | grep claude-3-sonnet

"anthropic.claude-3-sonnet-20240229-v1:0"

But yes, you need to apply for access.

jacoblee93 commented 7 months ago

Finished a PR, will merge and ship shortly!

bioshazard commented 7 months ago

Really appreciate the turnaround on this!!!

crileyy commented 7 months ago

@jacoblee93 Does your PR also fix this for using claude 3 with the Bedrock client (not the BedrockChat)? import { Bedrock } from 'langchain/llms/bedrock'

jacoblee93 commented 7 months ago

Claude-3 is messages only :( you'll need to use the chat model. It should be pretty interchangeable though?

langchain-ai / langchainjs