CachedContent.create() gives "403 You do not have permission to access the File" error for Video files.

[tobyperrett]

Description of the bug:

When I try to create a caching.CachedContent.create(), I get a "403 You do not have permission to access the File XXXX" error. I've checked and the model runs fine if I don't cache. The file does exist and is marked as "ACTIVE".

Actual vs expected behavior:

It was working a couple of days ago.

Any other information you'd like to share?

No response

[manojssmk]

Hi @tobyperrett

Could you share the sample code you're working with? This will help us reproduce the issue you've described.

Thanks

[barun-saha]

Hi,

I'm facing this issue as well. The sample code used and the error message are shown below.

# Source: https://ai.google.dev/gemini-api/docs/caching?lang=python#generate-content

import getpass
import os
import google.generativeai as genai
from google.generativeai import caching
import datetime
import time

# Get your API key from https://aistudio.google.com/app/apikey
# and access your API key as an environment variable.
# To authenticate from a Colab, see
# https://github.com/google-gemini/cookbook/blob/main/quickstarts/Authentication.ipynb
genai.configure(api_key=getpass.getpass('Gemini API_KEY: '))

# Download video file
#!curl -O https://storage.googleapis.com/generativeai-downloads/data/Sherlock_Jr_FullMovie.mp4

path_to_video_file = 'Sherlock_Jr_FullMovie.mp4'

# Upload the video using the Files API
video_file = genai.upload_file(path=path_to_video_file)

# Wait for the file to finish processing
while video_file.state.name == 'PROCESSING':
  print('Waiting for video to be processed.')
  time.sleep(2)
  video_file = genai.get_file(video_file.name)

print(f'Video processing complete: {video_file.uri}')

# Create a cache with a 5 minute TTL
cache = caching.CachedContent.create(
    model='models/gemini-1.5-flash-001',
    display_name='sherlock jr movie', # used to identify the cache
    system_instruction=(
        'You are an expert video analyzer, and your job is to answer '
        'the user\'s query based on the video file you have access to.'
    ),
    contents=[video_file],
    ttl=datetime.timedelta(minutes=5),
)

# Construct a GenerativeModel which uses the created cache.
model = genai.GenerativeModel.from_cached_content(cached_content=cache)

# Query the model
response = model.generate_content([(
    'Introduce different characters in the movie by describing '
    'their personality, looks, and names. Also list the timestamps '
    'they were introduced for the first time.')])

print(response.usage_metadata)

# The output should look something like this:
#
# prompt_token_count: 696219
# cached_content_token_count: 696190
# candidates_token_count: 214
# total_token_count: 696433

print(response.text)

Output:

% Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  316M  100  316M    0     0  63.0M      0  0:00:05  0:00:05 --:--:-- 69.9M
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Waiting for video to be processed.
Video processing complete: https://generativelanguage.googleapis.com/v1beta/files/ma9crud9vf6e

WARNING:tornado.access:403 POST /v1beta/cachedContents?%24alt=json%3Benum-encoding%3Dint (127.0.0.1) 736.81ms

---------------------------------------------------------------------------

Forbidden                                 Traceback (most recent call last)

[<ipython-input-35-5939ef08e240>](https://localhost:8080/#) in <cell line: 33>()
     31 
     32 # Create a cache with a 5 minute TTL
---> 33 cache = caching.CachedContent.create(
     34     model='models/gemini-1.5-flash-002',
     35     display_name='sherlock jr movie', # used to identify the cache

4 frames

[/usr/local/lib/python3.10/dist-packages/google/ai/generativelanguage_v1beta/services/cache_service/transports/rest.py](https://localhost:8080/#) in __call__(self, request, retry, timeout, metadata)
    408             # subclass.
    409             if response.status_code >= 400:
--> 410                 raise core_exceptions.from_http_response(response)
    411 
    412             # Return the response

Forbidden: 403 POST https://generativelanguage.googleapis.com/v1beta/cachedContents?%24alt=json%3Benum-encoding%3Dint: You do not have permission to access the File ma9crud9vf6e or it may not exist.

The output is the same if I switch to Flash 002.

[tobyperrett]

The sample code from https://ai.google.dev/gemini-api/docs/caching?lang=python#generate-content posted by barun-saha above fails. In addition to the flash models they tried, it has been failing for pro-001 and pro-002 for me.

[xdevfaheem]

I'm getting same error. It was working fine, a day before. But all of a sudden, it throws this. I couldn't get a solution anywhere. So i decided to create a issue and just found out it is the first one here. Someone please look into this.

Thanks for opening this up @tobyperrett

[xdevfaheem]

@tobyperrett is it resolved for you?

@manojssmk

[tobyperrett]

No, still broken.

[xdevfaheem]

It is somehow resolved now. idk what has been tweaked.

[barun-saha]

The cache creation appears to work, but now I get a different error when querying the model created with cached content:

BadRequest                                Traceback (most recent call last)

[<ipython-input-31-bf479f1bc745>](https://localhost:8080/#) in <cell line: 48>()
     46 
     47 # Query the model
---> 48 response = model.generate_content([(
     49     'Introduce different characters in the movie by describing '
     50     'their personality, looks, and names. Also list the timestamps '

9 frames

[/usr/local/lib/python3.10/dist-packages/google/ai/generativelanguage_v1beta/services/generative_service/transports/rest.py](https://localhost:8080/#) in __call__(self, request, retry, timeout, metadata)
    845             # subclass.
    846             if response.status_code >= 400:
--> 847                 raise core_exceptions.from_http_response(response)
    848 
    849             # Return the response

BadRequest: 400 POST https://generativelanguage.googleapis.com/v1beta/models/gemini-1.5-flash-002:generateContent?%24alt=json%3Benum-encoding%3Dint: Request contains an invalid argument.

[MarkDaoust]

Confirmed, video files + caching are intermittently failing for me too, Colab to reproduce:

https://colab.research.google.com/drive/1aqtluHSMhVKDzW96upKcE2oZYwpXMQqo

[manojssmk]

Hi @MarkDaoust

Now caching is working fine. Here is the gist

Thanks

[MarkDaoust]

Maybe fixed, please let me know if it comes back.