search

langchain-ai / langsmith-sdk

LangSmith Client SDK Implementations
https://smith.langchain.com/
MIT License
354 stars 59 forks source link

Test Run Exports #537

Open chasemcdo opened 3 months ago

chasemcdo commented 3 months ago

Feature request

The ability to export completed test runs in a similar manner to how you can export raw datasets.

Motivation

As a researcher or anyone hoping to present finding or evaluation metrics to other parties, it'd be great if these test results could be exported to a CSV file as can be done with raw datasets. I love the feature for datasets and is hopefully a small change to reuse that functionality for tests themselves.

I'm specifically working on a paper where we are hoping to use LLM evaluators for assessing different prompt versions and would love to be able to export the results from LangSmith for use in said paper.

Thanks!

hinthornw commented 3 months ago

Would either of these examples unblock you?

We plan to implement in the UI but have a few other priorities to resolve first before getting to this

chasemcdo commented 3 months ago

Yep, Thanks @hinthornw ! Would also be great to have a button in the ui eventually, but this does the trick for now. Thank you!

junyu commented 2 months ago

Hi - Sorry if this is not the right place to ask - I am trying to download the result of annotations (or feedback labels?) our human evaluators have marked in the Langsmith UI, but seems list_feedback doesn't include those results. Is this intended? Or maybe I am using it the wrong way?

hinthornw commented 2 months ago

list_feedback does include those! Do you have a code snippet?

junyu commented 1 month ago

list_feedback does include those! Do you have a code snippet?

Thank you for your assistance and sorry for the delayed response. The Langsmith library is now functioning properly after I updated it to the latest version.