openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.61k stars 784 forks source link

Updated Readme #171

Closed skfrost19 closed 8 months ago

skfrost19 commented 1 year ago
hauntsaninja commented 8 months ago

Closing, since it's been a few months without a response