[Feature]: Please add support for new model gemini/gemini-exp-1114 (maybe soon also on Vertex AI)

[Jasmin68k]

The Feature

Hello!

There's a new Gemini model available on https://ai.google.dev/gemini-api/docs/models/experimental-models. Please add.

On Vertex AI https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/gemini-experimental there still is the old experimental Gemini model. They might soon upgrade to the new one as well, in which case, please add that, too.

Motivation, pitch

Supporting new models is great.

Twitter / LinkedIn details

@Jasmin68k

[ishaan-jaff]

it should already work - what's the error @Jasmin68k ?

[Jasmin68k]

jasmin@workstation2 ~ $ litellm --model gemini/gemini-exp-1114
INFO:     Started server process [35228]
INFO:     Waiting for application startup.

#------------------------------------------------------------#
#                                                            #
#            'The thing I wish you improved is...'            #
#        https://github.com/BerriAI/litellm/issues/new        #
#                                                            #
#------------------------------------------------------------#

 Thank you for using LiteLLM! - Krrish & Ishaan

Give Feedback / Get Help: https://github.com/BerriAI/litellm/issues/new

INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:4000 (Press CTRL+C to quit)
00:39:55 - LiteLLM:WARNING: common_utils.py:33 - Unable to identify if system message supported. Defaulting to 'False'. Received error message - Model not supports system messages. You passed model=gemini-exp-1114, custom_llm_provider=gemini.
Add it here - https://github.com/BerriAI/litellm/blob/main/model_prices_and_context_window.json
INFO:     127.0.0.1:50614 - "POST /chat/completions HTTP/1.1" 200 OK
00:39:56 - LiteLLM Proxy:ERROR: proxy_server.py:2698 - litellm.proxy.proxy_server.async_data_generator(): Exception occured - litellm.NotFoundError: VertexAIException - b'{\n  "error": {\n    "code": 404,\n    "message": "models/gemini-exp-1114 is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.",\n    "status": "NOT_FOUND"\n  }\n}\n'
Traceback (most recent call last):
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/gemini/vertex_and_google_ai_studio_gemini.py", line 1032, in make_call
    response = await client.post(api_base, headers=headers, data=data, stream=True)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 162, in post
    raise e
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/llms/custom_httpx/http_handler.py", line 122, in post
    response.raise_for_status()
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/httpx/_models.py", line 763, in raise_for_status
    raise HTTPStatusError(message, request=request, response=self)
httpx.HTTPStatusError: Client error '404 Not Found' for url 'https://generativelanguage.googleapis.com/v1beta/models/gemini-exp-1114:streamGenerateContent?key=XXX&alt=sse'
For more information check: https://developer.mozilla.org/en-US/docs/Web/HTTP/Status/404

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1770, in __anext__
    await self.fetch_stream()
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1754, in fetch_stream
    self.completion_stream = await self.make_call(
                             ^^^^^^^^^^^^^^^^^^^^^
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/llms/vertex_ai_and_google_ai_studio/gemini/vertex_and_google_ai_studio_gemini.py", line 1036, in make_call
    raise VertexAIError(
litellm.llms.vertex_ai_and_google_ai_studio.common_utils.VertexAIError: b'{\n  "error": {\n    "code": 404,\n    "message": "models/gemini-exp-1114 is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.",\n    "status": "NOT_FOUND"\n  }\n}\n'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/proxy/proxy_server.py", line 2677, in async_data_generator
    async for chunk in response:
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/litellm_core_utils/streaming_handler.py", line 1981, in __anext__
    raise exception_type(
          ^^^^^^^^^^^^^^^
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 2125, in exception_type
    raise e
  File "/home/jasmin/.local/share/pipx/venvs/litellm/lib/python3.12/site-packages/litellm/litellm_core_utils/exception_mapping_utils.py", line 1163, in exception_type
    raise NotFoundError(
litellm.exceptions.NotFoundError: litellm.NotFoundError: VertexAIException - b'{\n  "error": {\n    "code": 404,\n    "message": "models/gemini-exp-1114 is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.",\n    "status": "NOT_FOUND"\n  }\n}\n'

[ishaan-jaff]

does a regular httpx / POST request to the endpoint work - without LiteLLM ?

[Jasmin68k]

Err, no, sorry, should have checked before:

curl https://generativelanguage.googleapis.com/v1beta/models/gemini-exp-1114:generateContent?key=$GEMINI_API_KEY     -H 'Content-Type: application/json'     -X POST     -d '{
      "contents": [{
        "parts":[{
          "text": "Hi."}]}]}'
{
  "error": {
    "code": 404,
    "message": "models/gemini-exp-1114 is not found for API version v1beta, or is not supported for generateContent. Call ListModels to see the list of available models and their supported methods.",
    "status": "NOT_FOUND"
  }
}