search

Scale3-Labs / langtrace

Langtrace 🔍 is an open-source, Open Telemetry based end-to-end observability tool for LLM applications, providing real-time tracing, evaluations and metrics for popular LLMs, LLM frameworks, vectorDBs and more.. Integrate using Typescript, Python. 🚀💻📊
https://langtrace.ai
GNU Affero General Public License v3.0
424 stars 33 forks source link

Feature-Request: Support LiteLLM #222

Open VfBfoerst opened 1 month ago

VfBfoerst commented 1 month ago

Hey :)
I wanted to ask for an integration of litellm as we use it as a gateway to different ai providers. It would be very cool to trace the litellm requests. Is there already a way to do so? Or can it be implemented?

karthikscale3 commented 1 month ago

Hey @VfBfoerst , we are working on litellm and we are hoping to release that support soon. Probably in about a week from now. I will keep you posted. Are there any specific litellm calls you are hoping to capture? would love to learn more so it meets your expectations.

VfBfoerst commented 1 month ago

Awesome! I would like to add langtrace as a somewhat middleware to the litellm proxy. So all tokens/metrics should be traced. Maybe on a api key basis to get statistics/metrics per key. All that to detect in the end bottlenecks, track usage, detect dysfunctional parts.

karthikscale3 commented 1 month ago

Thanks for this information. This is helpful. I will give you an update once we have the support live

karthikscale3 commented 1 month ago

@VfBfoerst - Just wanted to provide an update. We are still working through the support. We should have an update for you next week. Thanks for your patience.