openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.61k stars 785 forks source link

BOOL-1940: Refactor code to work with Python 3.7 #162

Closed Mainstay-Noah-Huppert closed 1 year ago

Mainstay-Noah-Huppert commented 1 year ago

BOOL-1940

The code path exercised by scripts/encode.py now works in Python 3.7.