search

dqbd / tiktoken

JS port and JS/WASM bindings for openai/tiktoken
MIT License
649 stars 49 forks source link

Add gpt-4 to model_to_encoding.json #25

Closed christophwitzko closed 1 year ago

christophwitzko commented 1 year ago

Hi there, thank you for creating this awesome package. 🎉

I added the encodings for all GPT-4 models.

Best, Chris

dqbd commented 1 year ago

Thanks @christophwitzko, lgtm 😄

christophwitzko commented 1 year ago

Thank you for merging! Can you also trigger a new npm release? 🙏 That would be amazing. ☺️

dqbd commented 1 year ago

Sure thing, 1.0.3 has been released!

Once again, thank you for your contribution @christophwitzko 👍

Fusseldieb commented 11 months ago

Just chiming in as it might help someone in the future.

I was doing tiktoken.get_encoding("gpt-4"); but it errored out saying "Error: Invalid encoding". Turns out I need to use: tiktoken.encoding_for_model("gpt-4");

Reading the docs help :)