search

voidKandy / espionox

Use espionox to streamline making LLM applications in Rust
https://crates.io/crates/espionox
10 stars 0 forks source link

Create a way of tracking tokens from GPT responses #3

Closed voidKandy closed 10 months ago

voidKandy commented 10 months ago

Should be as simple as adding tokens field to GptResponse struct, then assigning token -> dollar conversion based on current model

voidKandy commented 10 months ago

I have built token counts into the GptResponse struct, but I am still working out how to track them

voidKandy commented 10 months ago

Giving agents a running token use count might be a good way to keep track. All that would need to be added would be adding token counts from GptResponses returned from completion function to the running count of an agent

voidKandy commented 10 months ago

latest push to master adds usage field to GptResponse struct, and token_count to Gpt struct. Agent's prompt method updates Gpt's count when response is returned. I belive more work will be done in the future to make the process of storing and reading the token count 'better overall' but I'm closing the issue for now