langchain-community/chat_models/Bedrock [Feature]: Agent with Tools using Bedrock Anthropic Claude new tools API

[LordMsz]

Checked other resources

• [X] I added a very descriptive title to this issue.
• [X] I searched the LangChain.js documentation with the integrated search.
• [X] I used the GitHub search to find a similar question and didn't find it.
• [X] I am sure that this is a bug in LangChain.js rather than my code.
• [X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Example Code

import { config } from 'dotenv';
import { BedrockChat } from "@langchain/community/chat_models/bedrock";
// import { BedrockChat } from "./custom-models/index"; // this is my upadted version of BedrockChat that I want to propose as a PR
import { ChatPromptTemplate } from "@langchain/core/prompts";
import { z } from "zod";
import { createToolCallingAgent, AgentExecutor } from "langchain/agents";
import { DynamicStructuredTool } from "@langchain/core/tools";

config(); // load .env file

if (!process?.env?.AWSAccessKeyId || !process?.env?.AWSSecretAccessKey) {
    console.error('AWSAccessKeyId or AWSSecretAccessKey not found in .env file');
    process.exit(1);
}

const addTool = new DynamicStructuredTool({
    name: "add",
    description: "Add two integers together.",
    schema: z.object({
        firstInt: z.number(),
        secondInt: z.number(),
    }),
    func: async ({ firstInt, secondInt }) => {
        console.log('------ Function called with: -----------', { firstInt, secondInt });
        return (firstInt + secondInt).toString();
    },
});

async function test() {
    const model = new BedrockChat({
        model: "anthropic.claude-3-sonnet-20240229-v1:0",
        temperature: 0.5,
        region: "us-east-1",
        credentials: {
            accessKeyId: process.env.AWSAccessKeyId as string,
            secretAccessKey: process.env.AWSSecretAccessKey as string,
        },
    });

    const prompt = ChatPromptTemplate.fromMessages([
        ["system", "You are a helpful assistant"],
        ["placeholder", "{chat_history}"],
        ["human", "{input}"],
        ["placeholder", "{agent_scratchpad}"],
    ])

    const agent = createToolCallingAgent({
        llm: model,
        tools: [addTool],
        prompt: prompt,
        streamRunnable: false // attempt to work around streaming issue with tools
    });

    const agentExecutor = new AgentExecutor({ 
        agent,
        tools: [addTool],
        // verbose: true,
        returnIntermediateSteps: true,
    });

    const r = await agentExecutor.invoke({ input: "Hi! Can you tell me what is 10 + 4?" });
    console.log(r);
    console.log(r.content);
}

test();

Error Message and Stack Trace (if applicable)

This agent requires that the "bind_tools()" method be implemented on the input model.

Description

• I'm trying to use LangChain JS to create Tool Calling agent on Amazon Bedrock, using Claude 3 model with latest tools APi
• I get above error because in community/Bedrock this is not implemented yet
• I've finished this implementation on my own (mostly copying from Anthropic implementation) and would like to make a PR with it, and get support to finish it - as I don't have experience contributing to this project
• On my local test the function call gets executed as follows:

Binding tools to model [
  DynamicStructuredTool {
    lc_serializable: false,
    lc_kwargs: {
      name: 'add',
      description: 'Add two integers together.',
      schema: [ZodObject],
      func: [AsyncFunction: func]
    },
    lc_runnable: true,
    name: 'add',
    verbose: false,
    callbacks: undefined,
    tags: [],
    metadata: {},
    returnDirect: false,
    description: 'Add two integers together.',
    func: [AsyncFunction: func],
    schema: ZodObject {
      spa: [Function: bound safeParseAsync] AsyncFunction,
      _def: [Object],
      parse: [Function: bound parse],
      safeParse: [Function: bound safeParse],
      parseAsync: [Function: bound parseAsync] AsyncFunction,
      safeParseAsync: [Function: bound safeParseAsync] AsyncFunction,
      refine: [Function: bound refine],
      refinement: [Function: bound refinement],
      superRefine: [Function: bound superRefine],
      optional: [Function: bound optional],
      nullable: [Function: bound nullable],
      nullish: [Function: bound nullish],
      array: [Function: bound array],
      promise: [Function: bound promise],
      or: [Function: bound or],
      and: [Function: bound and],
      transform: [Function: bound transform],
      brand: [Function: bound brand],
      default: [Function: bound default],
      catch: [Function: bound catch],
      describe: [Function: bound describe],
      pipe: [Function: bound pipe],
      readonly: [Function: bound readonly],
      isNullable: [Function: bound isNullable],
      isOptional: [Function: bound isOptional],
      _cached: null,
      nonstrict: [Function: passthrough],
      augment: [Function: extend]
    }
  }
]
------ Function called with: ----------- { firstInt: 10, secondInt: 4 }
{
  input: 'Hi! Can you tell me what is 10 + 4?',
  output: '10 + 4 = 14',
  intermediateSteps: [ { action: [Object], observation: '14' } ]
}
undefined

System Info

npm info langchain:

langchain@0.2.4 | MIT | deps: 16 | versions: 276
Typescript bindings for langchain
https://github.com/langchain-ai/langchainjs/tree/main/langchain/

keywords: llm, ai, gpt3, chain, prompt, prompt engineering, chatgpt, machine learning, ml, openai, embeddings, vectorstores

dist
.tarball: https://registry.npmjs.org/langchain/-/langchain-0.2.4.tgz
.shasum: a85295e2510425cc5f4aca32bc9adeda88f6918d
.integrity: sha512-zBsBuNREn/3IlWvIQqhQ2iqf6JJhyjjsB1Db/keDkcgThPI3EcblC1pqAXU2BIKHmpNUkHBR2bAUok5+xtgOcw==
.unpackedSize: 4.0 MB

dependencies:
@langchain/core: ~0.2.0          js-tiktoken: ^1.0.12             langsmith: ~0.1.30               uuid: ^9.0.0
@langchain/openai: ~0.1.0        js-yaml: ^4.1.0                  ml-distance: ^4.0.0              yaml: ^2.2.1
@langchain/textsplitters: ~0.0.0 jsonpointer: ^5.0.1              openapi-types: ^12.1.3           zod-to-json-schema: ^3.22.3
binary-extensions: ^2.2.0        langchainhub: ~0.0.8             p-retry: 4                       zod: ^3.22.4

maintainers:
- nfcampos <nuno@boringbits.io>
- jacoblee93 <jacoblee93@gmail.com>
- andrewnguonly <andrewnguonly@gmail.com>
- davidduong <david@duong.cz>
- hwchase17 <hw.chase.17@gmail.com>
- basproul <braceasproul@gmail.com>

dist-tags:
latest: 0.2.4     next: 0.2.3-rc.0

published 3 days ago by jacoblee93 <jacoblee93@gmail.com>

Windows 11 64bit Node v20.14.0 No yarn atm

[LordMsz]

Relevant discussion link: https://github.com/langchain-ai/langchainjs/discussions/5610