langchain-ai / langsmith-sdk

LangSmith Client SDK Implementations
https://docs.smith.langchain.com/
MIT License
421 stars 80 forks source link

support downloading attachments so that they can be used for eval runs #1208

Open oegeshipster opened 1 week ago

oegeshipster commented 1 week ago

Feature request

although it is possible to upload files and view them in the dashboard, when i download the run they do not show up:

>>>run = client.read_run(run_id)
>>>right_run.attachments
{}

Motivation

especially for a workflow of input_file >> preprocessing >> llm extraction it is very useful to be able to go from the file input to the result in your eval experiments, instead of only from the preprocessed content to the result.

for example when I extract information from a pdf I would like to experiment both with different ocr providers and settings, and with different models and prompts. For that I need access to the original pdf not just the ocr extracted text in chat input format.