search

traceloop / openllmetry

Open-source observability for your LLM application, based on OpenTelemetry
https://www.traceloop.com/openllmetry
Apache License 2.0
1.56k stars 123 forks source link

🚀 Feature: cost metrics across all instrumentations #1042

Open nirga opened 2 months ago

nirga commented 2 months ago

Which component is this feature for?

All Packages

🔖 Feature description

Add span attributes and metrics for total cost for each model we instrument. Right now we only send token usage but we can easily extend this and send cost as well - we just need to define a prices const in each instrumentation and multiple the prompt/completion token usage.

🎤 Why is this feature needed ?

Better visibility into the cost for OpenLLMetry usage. Right now we only provide token usage which is basically just a proxy for cost.

✌️ How do you aim to achieve this?

See feature decsription

🔄️ Additional Information

No response

👀 Have you spent some time to check if this feature request has been raised before?

Are you willing to submit PR?

None

apuwk commented 2 months ago

Is this available to work on? Im going to go ahead and give it a shot if no one else is already working on it

kocobee commented 1 month ago

Might be able to re-use an existing mapping: https://github.com/openlit/openlit/blob/main/assets/pricing.json

kocobee commented 3 weeks ago

@apuwk any progress?

If not, I might take this.

nirga commented 3 weeks ago

@kocobee looks like he started working on it with #1303 but didn't continue. Let's wait for him to comment and if not maybe you can take it from there. You could also work on doing something similar to Anthropic or other frameworks - he hasn't done that and we'd love that as well :)

apuwk commented 3 weeks ago

@kocobee Hi, Yeah I'm still working on this and would like to keep working through it. But like Nir mentioned, you could start working on doing the same for other frameworks. Just update this chat with what you are working on if you can.