livekit / agents

Build real-time multimodal AI applications 🤖🎙️📹
https://docs.livekit.io/agents
Apache License 2.0
1.03k stars 198 forks source link

Add support for collecting usage data from plugin responses #613

Open willsmanley opened 1 month ago

willsmanley commented 1 month ago

I need to track token usage on a per-user and per-session basis as I'm sure many others do as well.

It would be nice to configure llm like this:

def token_usage_callback(usage_data):
    print(usage_data)

llm = openai.llm(
   model="gpt-4o", 
   token_usage_callback=token_usage_callback
)

and then you can store all of that usage data as needed for analytics and attribution

willsmanley commented 1 month ago

hi @theomonnom what do you think about enabling usage attribution1 from plugins like this? is this already on your roadmap?