search

microsoft / promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
https://microsoft.github.io/promptflow/
MIT License
9.49k stars 867 forks source link

[Feature Request] Enhanced filtering in tracing UI #3802

Open jomalsan opened 1 month ago

jomalsan commented 1 month ago

Is your feature request related to a problem? Please describe. I would like to be able to filter on more complex scenarios in the tracing UI. Some things I would love to see:

  1. Show me the points that failed to run successfully
  2. Show me the points where evaluation metric X is less than some threshhold
  3. Show me data points where output field X is Y
  4. Show me data points where the input field X contains Y

Describe the solution you'd like Standard type based filters applied to the user defined parts of the table: input, output, metrics

Describe alternatives you've considered I have been outputting all of the results (input, experiment and evaluation) to a jsonl file and then writing my own dataframe operations to identify the points of interest. I then have to go back to the tracing UI once I identify the point to see all of the intermediary steps.

github-actions[bot] commented 5 days ago

Hi, we're sending this friendly reminder because we haven't heard back from you in 30 days. We need more information about this issue to help address it. Please be sure to give us your input. If we don't hear back from you within 7 days of this comment, the issue will be automatically closed. Thank you!

jomalsan commented 4 days ago

bump