Implement token limit - Githubissues

namin / llm-verified-with-monte-carlo-tree-search

LLM verified with Monte Carlo Tree Search

https://arxiv.org/abs/2402.08147

MIT License

210 stars 25 forks source link

Implement token limit #44

Closed shenniger closed 3 months ago

shenniger commented 3 months ago

Example: ./run.py --token-limit=1000

Note that this currently (very naively) counts the raw amount of tokens, including special tokens, padding tokens, etc, which inflates the absolute number of tokens. Not sure if this is a good idea.