openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
12.48k stars 856 forks source link

Description of repository has a typo #270

Closed markusheimerl closed 7 months ago

markusheimerl commented 8 months ago

"tiktoken is a fast BPE tokeniser for use with OpenAI's models." should be "tiktoken is a fast BPE tokenizer for use with OpenAI's models."

hauntsaninja commented 7 months ago

Existing spelling gets the point across, it's okay if Americans have to think for an extra millisecond :-)