DOC: How to setup LangSmith threads in LangServe

[camba1]

Issue with current documentation:

The documentation indicates how to setup thread in LangChain using the SDK (https://docs.smith.langchain.com/monitoring/faq/threads)

from langchain_anthropic import ChatAnthropic
from langchain_core.prompts import ChatPromptTemplate
from langchain_core.messages import HumanMessage, AIMessage
from langchain_core.output_parsers import StrOutputParser
import uuid
model = ChatAnthropic(model="claude-3-haiku-20240307")
prompt = ChatPromptTemplate.from_messages([('placeholder', "{messages}")])
chain = prompt | model
messages = [HumanMessage(content="hi! I'm bob")]
config = {"metadata": {"conversation_id": str(uuid.uuid4())}}
response = chain.invoke({"messages": messages}, config=config)
messages = messages + [response, HumanMessage(content="whats my name")]
response = chain.invoke({"messages": messages}, config=config)

But how do you accomplish the same thing if the application is running in LangServe? My understanding is that you pass the chain to LangServe and it is LangServe that runs it when a request comes in. So, where would you add the thread configuration?

Idea or request for content:

Add another tab to the section where it explains how to set this up in python and LangChain Python with an example on how to accomplish this on LangServe.

[hinthornw]

Hi @camba1 - are you using RemoteRunnable? If so, you can pass in metadata the same way as you would here!

[camba1]

Oh, ok. So the client would be in charge of setting the correct metadata. Something like this:

    chain = RemoteRunnable("http://127.0.0.1:8000/simple/")
    config = {"metadata": {"conversation_id": str(uuid.uuid4())}}
    result = chain.invoke({"input": "Hello"}, config=config)

Cool. Thanks!

[nrakocz]

This does not work for me. I get multiple traces. One which contains the run but without the metadata. And another that contains just the metadata but without the run.

[ashantanu]

facing the same issue. using metadata or configurable config does not work