huggingface / blog

Public repo for HF blog posts
https://hf.co/blog
2.25k stars 706 forks source link

README claims of token size are outdated and inaccurate. #1653

Open Pixel-Panda opened 9 months ago

Pixel-Panda commented 9 months ago

It’s important to avoid ever using claims that are dependent on thousands of variables changing each day, such as, “With a context length of over 8,000 tokens, the StarCoder models can process more input than any other open LLM,” unless you’re going to fetch this using some logic that benchmarks daily. Or include a disclaimer footnote with the date of the claim on that same line. Already OpenAI models have surpassed this three times since this was written.

Km3888 commented 4 months ago