search

stanfordnlp / dspy

DSPy: The framework for programming—not prompting—foundation models
https://dspy-docs.vercel.app/
MIT License
13.82k stars 1.06k forks source link

dspy and langfuse integration #1179

Open xucailiang opened 1 week ago

xucailiang commented 1 week ago

Langfuse is a good prompt management project. I would like to know if dspy has plans to support langfuse. and I have written some code for this and am currently verifying them Thank you all🙏

lebsral commented 1 week ago

Are you suggesting something like saving the compiled optimized json programs that DSPy creates to langfuse? (at your prompt management). Or something else? Can you give some more details of what you see happening?

xucailiang commented 1 week ago

Are you suggesting something like saving the compiled optimized json programs that DSPy creates to langfuse? (at your prompt management). Or something else? Can you give some more details of what you see happening?

Yes, after the dspy optimization prompt, I hope to be able to easily observe what changes have been made. In this way, we can do some comparisons and cost statistics.

Of course, we can also use a similar method inspect_history, but it would be better if there is a more intuitive management page to observe.

Thank you for your reply.

denisergashbaev commented 1 week ago

That´s a good suggestion. On my side, I would want DSPy integration with Langfuse. That is, I want to see the calls that DSPy makes to LLM, grouped into traces and spans.

I see there was a PR for this: https://github.com/stanfordnlp/dspy/pull/200

xucailiang commented 1 week ago

That´s a good suggestion. On my side, I would want DSPy integration with Langfuse. That is, I want to see the calls that DSPy makes to LLM, grouped into traces and spans.

I see there was a PR for this: #200

I also noticed this PR, unfortunately，it was submitted on Oct 31, 2023, and has not been merged yet, dspy and langfuse have been updated for many versions.

So far, I have completed part of the code for chat_request and completions_request， If necessary, I will submit these changes and try to connect langfuse elsewhere.

arnavsinghvi11 commented 1 week ago

Feel free to build on #200 @xucailiang (will need to be updated with latest DSPy and/or Langfuse changes) or open a new PR for this integration!

xucailiang commented 1 week ago

Feel free to build on #200 @xucailiang (will need to be updated with latest DSPy and/or Langfuse changes) or open a new PR for this integration!

As you said, I have submitted the relevant code, PR thank you everyone🙏