search

google-gemini / generative-ai-python

The official Python library for the Google Gemini API
https://pypi.org/project/google-generativeai/
Apache License 2.0
1.45k stars 283 forks source link

Allow to modify multiple request options in Python SDK (similar to JavaScript SDK) #367

Open josealvarez97 opened 4 months ago

josealvarez97 commented 4 months ago

Description of the feature request:

I would like the Python SDK to allow the same equivalent functionality as the JavaScript SDK regarding the possibility of modifying the request options

JavaScript SDK (can add many custom request options)

const model = googleGenAIClient.getGenerativeModel(
                { model: "gemini-pro" },
                {
                    customHeaders: {
                        "x-param": "<PARAMETER>",
                    },
                    baseUrl: env.PROXY_URL,

                }
            );

PythonSDK (cannot add many custom request options)

import google.generativeai as genai

genai.configure(api_key="AIzaSyBW75RG6YjRor0pL1Eu0jWsRSd-rndTNIY")

model = genai.GenerativeModel('gemini-1.0-pro-latest')
response = model.generate_content("The opposite of hot is", 
                                  request_options= {
                                    # Only accepts timeout parameter
                                      "timeout": 10,
                                      # Anything else gives the following error
# TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'customHeaders'
                                    #   "customHeaders": {
                                    #         "x-param": "<PARAMETER>",
                                    #     },
                                    # I would also like to modify the baseUrl
                                    # "baseUrl": "http://localhost:8787/",
# TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'baseUrl'    
}
print(response.text)

What problem are you trying to solve with this feature?

Some people have talked about how being able to add metadata to requests in these libraries is helpful for proxies, and even tried to implement the solution themselves as you can find below:

I'm simply frustrated by the fact that it's possible to modify the request options in the JavaScript SDK, but not in the Python SDK.

If I can do it in the JavaScript SDK, why can't I do it in the Python SDK? 🤔

For the record, the Open AI Python SDK allows to do it by adding the parameter default_headers in the client configuration and extra_headers in the AI generation call:

from openai import OpenAI

client = OpenAI(
    api_key="<KEY>",
    base_url="http://localhost:8787",
    default_headers={
        "x-param": "parameter"
    }
)

chat_completion = client.chat.completions.create(
    messages=[
        {
            "role": "user",
            "content": "Say this is a test",
        }
    ],
    model="<model>",
    extra_headers={
        "x-param": "parameter"

    }
)

print(chat_completion.choices[0].message.content)

Any other information you'd like to share?

The documentation also seems to have implemented something similar to customHeaders in the request options (https://cloud.google.com/vertex-ai/generative-ai/docs/reference/nodejs/latest/vertexai/requestoptions), but when I installed the Python SDK, it didn't seem to accept customHeaders as explained above. At any rate, the current Python SDK is far behind the JavaScript SDK in this regard (JS SDK accepts other parameters like baseUrl—useful for proxies).

Additional context Add any other context or screenshots about the feature request here.

Other relevant links in this conversation may be:

bencipher commented 3 months ago

What’s the update on this. I can’t run a tagging Llm chain cos of this issue. I can create the chain but can’t invoke it, get the error below:

C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Scripts\python.exe C:\Users\BEN-UNKNOWN\PycharmProjects\Pet\brazillianscraper\scratch.py C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\llms__init__.py:548: LangChainDeprecationWarning: Importing LLMs from langchain is deprecated. Importing from langchain will no longer be supported as of langchain==0.2.0. Please import from langchain-community instead:

from langchain_community.llms import OpenAI.

To install langchain-community run pip install -U langchain-community. warnings.warn( C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain__init.py:29: UserWarning: Importing LLMChain from langchain root module is no longer supported. Please use langchain.chains.LLMChain instead. warnings.warn( C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core_api\deprecation.py:139: LangChainDeprecationWarning: The method Chain.run was deprecated in langchain 0.1.0 and will be removed in 0.3.0. Use invoke instead. warn_deprecated( Traceback (most recent call last): File "C:\Users\BEN-UNKNOWN\PycharmProjects\Pet\brazillianscraper\scratch.py", line 64, in main() File "C:\Users\BEN-UNKNOWN\PycharmProjects\Pet\brazillianscraper\scratch.py", line 59, in main print(chain.run(user_personal_details)) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core_api\deprecation.py", line 168, in warning_emitting_wrapper return wrapped(*args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 600, in run return self(args[0], callbacks=callbacks, tags=tags, metadata=metadata)[ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core_api\deprecation.py", line 168, in warning_emitting_wrapper return wrapped(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 383, in call return self.invoke( ^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 166, in invoke raise e File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\base.py", line 156, in invoke self._call(inputs, run_manager=run_manager) File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\llm.py", line 127, in _call response = self.generate([inputs], run_manager=run_manager) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain\chains\llm.py", line 139, in generate return self.llm.generate_prompt( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 677, in generate_prompt return self.generate(prompt_messages, stop=stop, callbacks=callbacks, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 534, in generate raise e File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 524, in generate self._generate_with_cache( File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_core\language_models\chat_models.py", line 749, in _generate_with_cache result = self._generate( ^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 760, in _generate response: GenerateContentResponse = _chat_with_retry( ^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 189, in _chat_with_retry return _chat_with_retry(kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity__init.py", line 330, in wrapped_f return self(f, *args, **kw) ^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity__init.py", line 467, in call do = self.iter(retry_state=retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity\init__.py", line 368, in iter result = action(retry_state) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity\init__.py", line 390, in self._add_action_func(lambda rs: rs.outcome.result()) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 449, in result return self.get_result() ^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN\AppData\Local\Programs\Python\Python311\Lib\concurrent\futures_base.py", line 401, in get_result raise self._exception File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\tenacity\init.py", line 470, in call__ result = fn(*args, kwargs) ^^^^^^^^^^^^^^^^^^^ File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 187, in _chat_with_retry raise e File "C:\Users\BEN-UNKNOWN.virtualenvs\jobfindr\brazvenv\Lib\site-packages\langchain_google_genai\chat_models.py", line 171, in _chat_with_retry return generation_method(kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ TypeError: GenerativeServiceClient.generate_content() got an unexpected keyword argument 'function_call'

Process finished with exit code 1