search

danny-avila / LibreChat

Enhanced ChatGPT Clone: Features Anthropic, AWS, OpenAI, Assistants API, Azure, Groq, o1, GPT-4o, Mistral, OpenRouter, Vertex AI, Gemini, Artifacts, AI model switching, message search, langchain, DALL-E-3, ChatGPT Plugins, OpenAI Functions, Secure Multi-User System, Presets, completely open-source for self-hosting. Actively in public development.
https://librechat.ai/
MIT License
18.9k stars 3.15k forks source link

Enhancement: Token count usage for Gemini models #3956

Open ss-gonda opened 1 month ago

ss-gonda commented 1 month ago

What happened?

When we call Gemini models from Vertex, no entry of usage is updated in the transaction collection.

Steps to Reproduce

  1. Deploy Gemini pro and/or vision pro in Vertex AI.
  2. Configure the settings to enable "google" endpoint and "gemini-pro" models.
  3. Login to the frontend and select Google endpoint with any model and make multiple chats.
  4. Note that no entry is updated in the transaction collection of Mongo.
  5. It is direct call to Google Vertex and is not through proxy.

Note that we have other models such as Anthropic (direct), Llama (litellm + Model Garden), AzureOpenAI and for all these models we get proper usage from the transaction collection. Only discrepancy is with Gemini models on Vertex.

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Screenshots

No response

Code of Conduct

danny-avila commented 1 month ago

Thanks. Not a bug, needs to be implemented