Add option to set a client for GenerativeModel

[kartik727]

Description of the feature request:

Currently, the Python Quickstart documentation for using the Gemini API suggests setting the API key in the default client configuration.

import google.generativeai as genai

genai.configure(api_key=GOOGLE_API_KEY)
model = genai.GenerativeModel('gemini-pro')
response = model.generate_content("What is the meaning of life?")
print(response.text) # Model response gets printed

In addition to this, the package should also give an option to set a different key when creating the model. Something like:

model = genai.GenerativeModel('gemini-pro', api_key=OTHER_GOOGLE_API_KEY)

Currently, a workaround is to manually set the private _client variable for the model

import google.generativeai as genai
from google.ai import generativelanguage as glm

client = glm.GenerativeServiceClient(
    client_options={'api_key':OTHER_GOOGLE_API_KEY})
model = genai.GenerativeModel('gemini-pro')
model._client = client
response = model.generate_content("What is the meaning of life?")
print(response.text) # Model response gets printed

What problem are you trying to solve with this feature?

The current implementation requires the API key to be set globally. Allowing the keys to be set with a smaller scope will be useful for applications that use multiple API keys concurrently for different tasks.

Any other information you'd like to share?

One potential implementation for this feature could be to take an additional keyword argument for the client and set it as the _client during the __init__ method of the GenerativeModel class.

# google/generative_ai/generative_models.py
class GenerativeModel:
    def __init__(
        self,
        model_name: str = "gemini-m",
        safety_settings: safety_types.SafetySettingOptions | None = None,
        generation_config: generation_types.GenerationConfigType | None = None,
        client = None, # Additional kwarg
    ):
    ...
    self._client = client
    ...

Then, the user can optionally supply the client with their API key if they do not want the default credentials to be used.

# main.py
import google.generativeai as genai
from google.ai import generativelanguage as glm

client = glm.GenerativeServiceClient(
    client_options={'api_key':OTHER_GOOGLE_API_KEY})
model = genai.GenerativeModel('gemini-pro', client=client)
response = model.generate_content("What is the meaning of life?")
print(response.text) # Model response gets printed

[LukeSamkharadze]

+1

[markmcd]

We can see how his is useful - let's leave it open to see how much interest there is in the feature and use that for prioritization.

[Tiendil]

We can see how his is useful - let's leave it open to see how much interest there is in the feature and use that for prioritization.

I think the feature "allows creating clients with different credentials" is a best practice for libs, if not a must-have.

Without such a feature, generative-ai-python adds too strong architectural restrictions to all projects that work with multiple users/companies. I speak about restrictions like "you must separate users by processes" that significantly complicate deployment, infrastructure, user management, etc. Especially, remembering that Python is good at single-thread concurrency with async/await mechanism.

For example, in my project, I want to allow users to set up their own API keys to process their data. Therefore, I'm forced to use the hack from the first message or complicate the architecture.

I believe this logic is valid for a lot of multi-tenant cases (especially built by small teams), not only for my project.

Also, the lack of per-client credentials complicates tracking costs/API usage per-use-case even in a single tenant environment.

[gordonhart]

+1 to the original post and @Tiendil's latest comment. Creating clients with different credentials within the same process is more or less essential for multitenant apps. As far as I can tell, every other major LLM Python client supports this through a simple client = Client(api_key=...) API.

Running into issues with google.generativeai being the odd duck by not supporting this pattern in my project AutoArena. This makes it cumbersome to include Gemini as one of our supported model families, which I would love to do!