Google Gen AI: Context caching breaks with generateObject / generateContent -

[chanmathew]

Description

Hi there,

I'm trying to implement context caching with the Gemini models, but it keeps returning an error saying:

CachedContent can not be used with GenerateContent request setting system_instruction, tools or tool_config. Proposed fix: move those values to CachedContent from GenerateContent request.

But I'm not passing in system_instruction, tools, or tool_config in my request. Not sure if I'm doing something wrong here?

export const extractedPlacesSchema = z.object({
    results: z.array(
        z.object({
            name: z.string(),
            link: z.string().nullable(),
            image: z.string().nullable(),
            city: z.string().nullable(),
            country: z.string().nullable()
        })
    )
});

const newCache = await cacheManager.create({
        model,
        displayName: crypto.randomUUID(),
        systemInstruction: prompt,
        contents: [
            {
                role: 'user',
                parts: [{ text }]
            }
        ],
        ttlSeconds: 60
    });

const { object } = await generateObject({
    model: google(model, {
        cachedContent: chunkCachedName,
        safetySettings
    }),
    temperature: 0,
    schema: extractedPlacesSchema,
    prompt: 'Extract from the provided content.'
});

It seems to work fine with generateText, however as soon as I use generateObject or generateContent it fails with that error.

Also just to clarify, if the system prompt is already included in the cacheManager, I don't need to specify it again in the generateObject right?

Code example

Here is the full error, you can see in the requestBodyValues there is a systemInstruction object being passed, potentially being injected by the SDK? perhaps that is the issue?

 Error on attempt 3/3: APICallError [AI_APICallError]: CachedContent can not be used with GenerateContent request setting system_instruction, tools or tool_config.

 Proposed fix: move those values to CachedContent from GenerateContent request.
     at file:///Users/mathew/Dev/talewind-app/node_modules/@ai-sdk/provider-utils/dist/index.mjs:431:14
     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
     at async postToApi (file:///Users/mathew/Dev/talewind-app/node_modules/@ai-sdk/provider-utils/dist/index.mjs:336:28)
     at async GoogleGenerativeAILanguageModel.doGenerate (file:///Users/mathew/Dev/talewind-app/node_modules/@ai-sdk/google/dist/index.mjs:364:50)
     at async fn (/Users/mathew/Dev/talewind-app/node_modules/ai/dist/index.mjs:2049:33)
     at async eval (/Users/mathew/Dev/talewind-app/node_modules/ai/dist/index.mjs:299:22)
     at async _retryWithExponentialBackoff (/Users/mathew/Dev/talewind-app/node_modules/ai/dist/index.mjs:129:12)
     at async fn (/Users/mathew/Dev/talewind-app/node_modules/ai/dist/index.mjs:2017:34)
     at async eval (/Users/mathew/Dev/talewind-app/node_modules/ai/dist/index.mjs:299:22)
     at async processChunk (/Users/mathew/Dev/talewind-app/apps/creator-app/src/lib/server/crawl.server.ts:224:24) {
   cause: undefined,
   url: 'https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-002:generateContent',
   requestBodyValues: {
     generationConfig: {
       topK: undefined,
       maxOutputTokens: undefined,
       temperature: 0,
       topP: undefined,
       stopSequences: undefined,
       responseMimeType: 'application/json',
       responseSchema: [Object]
     },
     contents: [ [Object] ],
     systemInstruction: { parts: [Array] },
     safetySettings: [ [Object], [Object], [Object], [Object] ],
     cachedContent: 'cachedContents/h7yj08xq8pe3'
   },
   statusCode: 400,
   responseHeaders: {
     'alt-svc': 'h3=":443"; ma=2592000,h3-29=":443"; ma=2592000',
     'cache-control': 'private',
     'content-encoding': 'gzip',
     'content-type': 'application/json; charset=UTF-8',
     date: 'Tue, 22 Oct 2024 18:20:08 GMT',
     server: 'scaffolding on HTTPServer2',
     'server-timing': 'gfet4t7; dur=84',
     'transfer-encoding': 'chunked',
     vary: 'Origin, X-Origin, Referer',
     'x-content-type-options': 'nosniff',
     'x-frame-options': 'SAMEORIGIN',
     'x-xss-protection': '0'
   },
   responseBody: '{\n' +
     '  "error": {\n' +
     '    "code": 400,\n' +
     '    "message": "CachedContent can not be used with GenerateContent request setting system_instruction, tools or tool_config.\\n\\nProposed fix: move those values to CachedContent from GenerateContent request.",\n' +
     '    "status": "INVALID_ARGUMENT"\n' +
     '  }\n' +
     '}\n',
   isRetryable: false,
   data: {
     error: {
       code: 400,
       message: 'CachedContent can not be used with GenerateContent request setting system_instruction, tools or tool_config.\n' +
         '\n' +
         'Proposed fix: move those values to CachedContent from GenerateContent request.',
       status: 'INVALID_ARGUMENT'
     }
   },
   [Symbol(vercel.ai.error)]: true,
   [Symbol(vercel.ai.error.AI_APICallError)]: true
 }

Additional context

"@ai-sdk/google": "^0.0.51", "ai": "^3.4.18", "@google/generative-ai": "^0.21.0",

[lgrammel]

I can see how this happens with tool mode. Can you try mode: 'json' and see if that helps?

[chanmathew]

@lgrammel Added that but still same error unfortunately.