search

parkervg / blendsql

Query language for blending SQL logic and LLM reasoning across structured + unstructured data. [Findings of ACL 2024]
https://parkervg.github.io/blendsql/
Apache License 2.0
72 stars 4 forks source link

Load Ollama and Llama.cpp tokenizers in `Model` class #14

Open parkervg opened 5 months ago

parkervg commented 5 months ago

TransformerLLM and OpenAILLM classes have a tokenizer attribute, which allows us to log prompt_tokens and completion_tokens.

We want the same for OllamaLLM and LlamaCppLLM