search

openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.61k stars 784 forks source link

ValueError: Unknown encoding cl100k_base error when program is compiled #168

Closed tomhayw closed 1 year ago

tomhayw commented 1 year ago

Running the program in an IDE works fine but when I compile it I am hit with this error: ValueError: Unknown encoding cl100k_base

How do I fix this?

hauntsaninja commented 1 year ago

By "compile" do you mean PyInstaller? See the FAQ https://github.com/openai/tiktoken/issues/98