search

enricoros / big-AGI

Generative AI suite powered by state-of-the-art models and providing advanced AI/AGI functions. It features AI personas, AGI functions, multi-model chats, text-to-image, voice, response streaming, code highlighting and execution, PDF import, presets for developers, much more. Deploy on-prem or in the cloud.
https://big-agi.com
MIT License
5.47k stars 1.26k forks source link

Usage meter & Cost Estimates #341

Open vadimkatsman opened 9 months ago

vadimkatsman commented 9 months ago

If to inspect the JSON file with exported conversations, you could see the token cost for both inbound (prompts) and outbound (generated text) messages. But that information is not available on the human interface.

Concise description To have an aggregated usage meter as well as chat / message cost counter. Since pricing for inbound and outbound tokens differ, to have 2 separate counts in the format like 300/240 (and maybe using different colors - green vs red etc.)

enricoros commented 9 months ago

Need more details to understand if this applies only to the last message, or former messages: Since once generated (output) message then becomes an input message for all the rest of the conversation, what's a json structure that would be able to capture this?

vadimkatsman commented 9 months ago

Your internal JSON includes the cost of each message. My suggestion is simply to display it on UX - I am not suggesting to compute it any different. With added aggregation across the chat and across all chats.

arbiterFF commented 9 months ago

I love this feature. It would be great to have this somewhere. Though not a top priority IMO (at least relative to moving architecture to cloud and being able to save and personas and chats x-device, or the ability to integrate with G-suite to RAG personal data)