Closed ziziele closed 1 year ago
You'll need to calculate the number with a library like tiktoken which calculates the number of tokens in a string (or you can get a rough count by counting the number of spaces and punctuation).
Here's a node module binding tiktoken: https://www.npmjs.com/package/@dqbd/tiktoken
You'll need to calculate the number with a library like tiktoken which calculates the number of tokens in a string (or you can get a rough count by counting the number of spaces and punctuation).
Here's a node module binding tiktoken: https://www.npmjs.com/package/@dqbd/tiktoken