missing TiktokenModel for version 3.5-turbo-16k

dqbd / tiktoken

JS port and JS/WASM bindings for openai/tiktoken

MIT License

648 stars 49 forks source link

Open hannahlaek-wt opened 9 months ago

hannahlaek-wt commented 9 months ago

missing TiktokenModel for version 3.5-turbo-16k

dqbd commented 9 months ago

Hey @hannahlaek-wt!

Thanks for flagging! Will add the missing model, but for now consider using cl100k_base directly.

Tanakpek commented 3 weeks ago

what would be the closest to 3.5 turbo instruct?