search

microsoft / promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
https://microsoft.github.io/promptflow/
MIT License
9.47k stars 864 forks source link

[Feature Request] Support external Tracing Tools (e.g. langfuse) #3573

Open carlFsq opened 3 months ago

carlFsq commented 3 months ago

Is your feature request related to a problem? Please describe. I want to trace my whole prompt flow application with my existing langfuse solution. This functions of course well with the python tool, but not with e.g. the LLM and the Embedding Tool.

Describe the solution you'd like It would be nice to use the existing prompt flow without just being able to use the Python Tool.

Describe alternatives you've considered Only using the Python Tool and passing the trace id to every node in the graph

Additional context

clemra commented 3 months ago

Hi all, Langfuse founder/maintainer here.

We'd love to support the promptflow community in building this out. We won't have internal capacity to build/maintain this in the coming weeks but can provide guidance and answer questions (here or via discord/slack channel as things move along).

Happy to pick this up if folks are interested. Wrapping this in our Python Decorator may be a possibility.

0mza987 commented 3 months ago

Hi @carlFsq @clemra ,

Thanks for your input and the time you took to share your idea with us.

Currently, we don't have the bandwidth to implement this feature right away, so we've labeled it as a long-term task. We hope to address it in the future as our resources allow.

tuanardouin commented 1 month ago

This would be wonderful.

We're currently building a custom integration, it's not ready yet, but as soon as it is we could try to add it here.

@aiancheruk @stephaneBylaw

clemra commented 1 month ago

Hi @tuanardouin -- that sounds great. Feel free to ping me if the Langfuse team can be of help. here or clemens at langfuse.com