Support for new "gpt-4o" tiktoken tokenizer

IAPark / tiktoken_ruby

Unofficial ruby binding for tiktoken by way of rust

MIT License

109 stars 26 forks source link

Closed gtoubassi closed 1 month ago

gtoubassi commented 1 month ago

I may take a run at it myself but happy to "buy someone a coffee" if they beat me to it :). Curious if there are any potential pitfalls.

IAPark commented 1 month ago

Looks like @CaffeeLake did this. Let me know if it doesn't work for you now

gtoubassi commented 1 month ago

@CaffeeLake thank you!!!!