feat: count tokens and token costs - Githubissues

Probable-Futures / climate-change-assistant

An experimental LLM agent to help people understand climate change. Not ready for use in a production environment.

0 stars 1 forks source link

feat: count tokens and token costs #22

Closed MoustafaWehbe closed 9 months ago

MoustafaWehbe commented 9 months ago

Description

Count tokens for each input and output message using tiktoken from openai.
Calculate the price of those token when a thread run is complete.
The price will only be calculated If the environment is local or dev
A message will be sent to the UI from the author "System" containing the total cost in the thread so far.
The cost is an estimate the AI could use much more tokens to choose which content from the thread to include while generating an answer

Screenshot/GIF