openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.61k stars 784 forks source link

Update README.md to clarify context of models #203

Closed logankilpatrick closed 7 months ago