langchain-ai / langsmith-sdk

LangSmith Client SDK Implementations
https://smith.langchain.com/
MIT License
346 stars 59 forks source link

Allow option to only trace if there is an exception #828

Closed NathanHam16 closed 1 day ago

NathanHam16 commented 5 days ago

Feature request

Allow conditional tracing, only if an exception has been thrown.

Motivation

For some LLM applications, there may be hundreds of API calls being made per single use case, due to the rising popularity of agentic workflows that focus on small, specific tasks. It would be great to trace only errors being thrown, and this would work great for other functions too.

hinthornw commented 1 day ago

That would add a lot of complexity if we implemented this client-side.

Right now, you can edit TTLs of traces in langsmith and then make it so errored runs have longer TTLs. This server-side handling is more reliable.

Handling client side would mean we'd have to keep an entire long trace in memory until it completes and then only send if there's an error. When the application is distributed, this because pretty fraught (the called process wouldn't be responding likely.

hinthornw commented 1 day ago

Going to move to discussions since this isn't a bug.