openai / tiktoken

tiktoken is a fast BPE tokeniser for use with OpenAI's models.
MIT License
11.61k stars 784 forks source link

Calculate token consumption for OpenAI function definitions #160

Closed Pwuts closed 1 year ago

Pwuts commented 1 year ago
hauntsaninja commented 1 year ago

This is out of scope for tiktoken, maybe file an issue on openai-python