search

AgentOps-AI / agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks like CrewAI, Langchain, and Autogen
https://agentops.ai
MIT License
2.06k stars 200 forks source link

Add DSPY instrumentor #88

Open areibman opened 8 months ago

areibman commented 8 months ago

🚀 Feature Request

💡 Got a brilliant idea? Using either OpenInference or OpenLLMetry, add an instrumented for DSPY. This is the first of several test features we'll use to determine which is the best instrumentation library.

Dsantra92 commented 7 months ago

This looks like a very cool idea, can I work on it?

areibman commented 7 months ago

This looks like a very cool idea, can I work on it?

Sure, feel free to take a stab. Are you using DSPY on the regular? Would love to know why this in particular seems interesting.

One option is to look at what other folks are doing. We saw OpenInference built a DSPY adapter already. . The idea would be to copy that pattern and either a) copy + adjust the code to create AgentOps events or b) create our own open telemetry collector and just outfit AgentOps to send data to that service.

It's really more of an experimental thing for us.

therealtimex commented 3 months ago

I see this on Langwatch: https://docs.langwatch.ai/dspy-visualization/quickstart

areibman commented 3 months ago

I see this on Langwatch: https://docs.langwatch.ai/dspy-visualization/quickstart

Could probably fork the integration approach they used