Can we add a way to estimate token size for Ollama?

griptape-ai / griptape

Modular Python framework for AI agents and workflows with chain-of-thought reasoning, tools, and memory.

https://www.griptape.ai

Apache License 2.0

1.95k stars 160 forks source link

Can we add a way to estimate token size for Ollama? #1151

Open shhlife opened 3 weeks ago

shhlife commented 3 weeks ago

[x ] I have read and agree to the contributing guidelines.

Currently we have a TokenCounter utility that will count tokens for drivers other than Ollama. Ollama currently doesn't provide token information in it's responses: https://github.com/ollama/ollama/issues/1716, but we could perhaps use a tokenizer to estimate token size?

This might be useful as well for customers to estimate a call before they run it.