Use appropriate tokenizer - Githubissues

ray-project / llmperf

LLMPerf is a library for validating and benchmarking LLMs

Apache License 2.0

659 stars 113 forks source link

Use appropriate tokenizer #68

Closed markVaykhansky closed 2 months ago