Closed lightningRalf closed 3 months ago
Counting tokens for Gemini models requires an API call, they do not have their tokenizer open-sourced / locally usable like tiktoken from OpenAI. This would require the user to configure their API key and the like adding more complexity.
The token counter in code2prompt is intended to be an indicator of how long the rendered prompt is, this should be enough to gauge the possibility of usage for any LLM models/tokenizers. Closing this feature request, for now, will reconsider for the next release candidates.
Hello, thanks for creating this! It looks super interesting. With Gemini having 1 million tokens available. And the possibility to get 2 million soon, it will be amazing to see the amount of tokens for them. https://ai.google.dev/gemini-api/docs/get-started/tutorial?lang=web#count-tokens
Thanks in advance for considering! Greetings from Franconia (Germany)