400 INVALID_ARGUMENT error when attempting to use Function Calling with Vertex AI Search retrieval tool

[dtrenz]

Environment details

• Programming language: TS/JS
• OS: macOS
• Language runtime version: Typescript 5.6.2
• Package version: 1.7.0

Steps to reproduce

1. Configure a Vertex AI generative model (getGenerativeModel) with a function calling tool and a Vertex AI Search retrieval tool:

   tools: [
     {
       functionDeclarations: [
         {
           name: "get_current_weather",
           description: "get weather in a given location",
           parameters: {
             type: FunctionDeclarationSchemaType.OBJECT,
             properties: {
               location: { type: FunctionDeclarationSchemaType.STRING },
             },
             required: ["location"],
           },
         },
       ],
       retrieval: {
         vertexAiSearch: {
           datastore: "<datastore>",
         },
         disableAttribution: false,
       },
     },
   ],

2. Create a ChatSession using startChat()
3. Call sendMessageStream on a ChatSession

Error output

 ⨯ Internal error: ClientError: [VertexAI.ClientError]: got status: 400 Bad Request. {"error":{"code":400,"message":"Request contains an invalid argument.","status":"INVALID_ARGUMENT"}}
    at throwErrorIfNotOK (./node_modules/@google-cloud/vertexai/build/src/functions/post_fetch_processing.js:32:27)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async generateContentStream (./node_modules/@google-cloud/vertexai/build/src/functions/generate_content.js:97:5)
digest: "3774699693"
Cause: GoogleApiError: Request contains an invalid argument.
    at throwErrorIfNotOK (webpack-internal:///(action-browser)/./node_modules/@google-cloud/vertexai/build/src/functions/post_fetch_processing.js:32:66)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async generateContentStream (webpack-internal:///(action-browser)/./node_modules/@google-cloud/vertexai/build/src/functions/generate_content.js:97:5) {
  code: 400,
  status: 'INVALID_ARGUMENT',
  errorDetails: undefined
}

[marius-ciocoiu]

There is a similar issue with googleSearchRetrieval, which can be resolved by including only the googleSearchRetrieval tool in the tools array. If we include any other tool besides that, we encounter a 400 INVALID_ARGUMENT error. It appears that when grounding is being used, we cannot currently use any other tool.