openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.06k stars 749 forks source link

Add turbo 16k model for Azure #256

Closed kartikagrawal2503 closed 4 months ago

kartikagrawal2503 commented 4 months ago

Azure OpenAI service allows us to deploy 16k token context version of gpt-turbo-3.5 model which has the name as gpt-35-turbo-16k.

kartikagrawal2503 commented 4 months ago

Thanks. Was using older version didn't noticed.