Request: Add prompt_billable_characters in gemini-pro

[abetatos]

When using VertexAI and ChatVertexAI you only get billable characters in models such as text-bison and chat-bison but not in gemini-pro. Why is that happening? Is there another way to obtain number of characters in order to estimate the cost of the service? I think cost is obtained by number of chars and not tokens, so I am a little confused.

Code:

from langchain_google_vertexai import ChatVertexAI
chat = ChatVertexAI(model_name=model_name, temperature=0, response_validation=False)
res = chat([HumanMessage(content="Hello")])

With gemini-pro

{'prompt_token_count': 1, 'candidates_token_count': 23, 'total_token_count': 24}

With chat-bison

{'candidates_billable_characters': 26.0, 'candidates_token_count': 9.0, 'prompt_billable_characters': 5.0, 'prompt_token_count': 1.0}

I would like to know if this could be added withouth the necessity of calling the free endpoint explained in here: https://cloud.google.com/vertex-ai/generative-ai/docs/multimodal/get-token-count. I wouldn't like to add latency to the system.

Thank you very much!

[lkuligin]

Gemini doesn't return characters yet as part of the response.

[abetatos]

Will it return them in the future? Do you know anything about it?

[antti-ngp]

I guess gemini will be billed by tokens starting today: https://ai.google.dev/pricing