Unable to use Gemini stable model for caching in vertexai preview

[aminecs]

Hi team,

I am trying to use context caching in Gemini through vertexai, however, when trying to create the content to cache, it fails as it can't find the stable model.

I can see that context caching has been moved back to public preview (#3935) so it should be accessible.

Environment details

• google-cloud-aiplatform version: 1.56.0

Steps to reproduce

from vertexai.preview import caching
cache = caching.CachedContent.create(
    model_name='publishers/google/models/gemini-1.5-pro-001',
    system_instruction=(
        'random'
    ),
    contents=[prompt],
    ttl=datetime.timedelta(minutes=5),
)

Stack trace

400 Model ****/*****/locations/europe-west1/publishers/google/models/gemini-1.5-pro-001 is not supported for caching. Please use the stable models that are enabled for caching.

[ZhenyiQ]

Thanks @aminecs for reporting this! I was able to reproduce it and I'm contacting the service team.

[Mojashi]

The situation seems different between the two models and regions.

• gemini-1.5-pro-001: I could create cache and use the cache in "us-central-1", but not in other regions.
• gemini-1.5-flash-001: I could create cache in "us-central-1" but cannot use it in anywhere.

[aminecs]

@Mojashi Unfortunately I am unable to use other regions. Due to specific requirements, I need to keep the requests within Europe.

Please keep me posted if you have updates @ZhenyiQ