I would like to add a new feature that will allow users to see token usage information when interacting with LLMs. This information includes:
Completion tokens
Prompt tokens
Total tokens used
The new feature would work when the program is run with the --token-usage or -t flag. Token usage information should be displayed in stderr, and it will be helpful in understanding the number of tokens consumed during a prompt and completion cycle.
The output might look something like this:
completion_tokens: XX
prompt_tokens: YY
total_tokens: ZZ
I would like to add a new feature that will allow users to see token usage information when interacting with LLMs. This information includes:
--token-usageor-tflag. Token usage information should be displayed in stderr, and it will be helpful in understanding the number of tokens consumed during a prompt and completion cycle.The output might look something like this: completion_tokens: XX prompt_tokens: YY total_tokens: ZZ