openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
12.48k stars 856 forks source link

Add turbo 16k model for Azure #256

Closed kartikagrawal2503 closed 9 months ago

kartikagrawal2503 commented 9 months ago

Azure OpenAI service allows us to deploy 16k token context version of gpt-turbo-3.5 model which has the name as gpt-35-turbo-16k.

kartikagrawal2503 commented 9 months ago

Thanks. Was using older version didn't noticed.