Open manvkaur opened 4 months ago
OpenAI provides a Python library called tiktoken that can be used to count tokens in a text string according to OpenAI's tokenization rules. Unfortunately, since C# is not directly supported by OpenAI, won't be able to use the tiktoken library directly in C#; will need custom implementation.
It looks like there are some 3rd party MIT-licensed libraries we could consider:
In the current implementation for TotalTokens in ChatCompletion.
Initialise the chat state with PUT method
Retrieve the chat state with GET method (TotalTokens is 0) because nothing has been posted to OpenAI yet and it has not returned consumed tokens.
GET http://localhost:7071/api/chats/test123?timestampUTC=2023-08-10T07:51:10Z
Behavior: Actual: The response returns totalTokens to be 0. Expected: it should count the instructions tokens.