search

openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
12.48k stars 856 forks source link

Junhyun/add upstage solar #244

Closed jhpark-upstage closed 10 months ago