KeyError with RunnableWithMessageHistory and LangServe: Missing Variable

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.
[X] The bug is not resolved by updating to the latest stable version of LangChain (or the specific integration package).

Description

When starting langserve with the code below and accessing it via RemoteRunnable, I encounter a KeyError: "Input to ChatPromptTemplate is missing variables {'language'}. Expected: ['history', 'input', 'language'] Received: ['input', 'history']".

Example Code

from langchain_openai import ChatOpenAI
from langchain_core.prompts import ChatPromptTemplate, MessagesPlaceholder
from langchain_core.runnables.history import RunnableWithMessageHistory
from langchain_core.chat_history import BaseChatMessageHistory
from langchain_community.chat_message_histories import ChatMessageHistory
from langserve import add_routes
from fastapi import FastAPI
import uvicorn

app = FastAPI()

store = {}

def get_session_history(session_id: str) -> BaseChatMessageHistory:
    if session_id not in store:
        store[session_id] = ChatMessageHistory()
    return store[session_id]

prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            "You're an assistant who speaks in {language}. Respond in 20 words or fewer.",
        ),
        MessagesPlaceholder(variable_name="history"),
        ("human", "{input}"),
    ]
)

model = ChatOpenAI(model="gpt-3.5-turbo-0125")

runnable = prompt | model

runnable_with_history = RunnableWithMessageHistory(
    runnable,
    get_session_history,
    input_messages_key="input",
    history_messages_key="history",
)

chain = runnable_with_history

add_routes(app, chain, path="/test")

uvicorn.run(app, host="0.0.0.0", port=8000)

Code for RemoteRunnable:

from langserve import RemoteRunnable

rr = RemoteRunnable("http://localhost:8000/test/")
rr.invoke(
    {"language": "Italian", "input": "what's my name?"},
    config={"configurable": {"session_id": "1"}},
)

This issue also occurs in the LangServe Playground where the input box for language does not appear. When sending the message as-is, it results in KeyError: "Input to ChatPromptTemplate is missing variables {'language'}. Expected: ['history', 'input', 'language'] Received: ['input', 'history']".

Error Message and Stack Trace (if applicable)

INFO: 127.0.0.1:57555 - "POST /test/invoke HTTP/1.1" 500 Internal Server Error ERROR: Exception in ASGI application Traceback (most recent call last): File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/uvicorn/protocols/http/httptools_impl.py", line 426, in run_asgi result = await app( # type: ignore[func-returns-value] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/uvicorn/middleware/proxy_headers.py", line 84, in call return await self.app(scope, receive, send) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/fastapi/applications.py", line 1054, in call await super().call(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/applications.py", line 123, in call await self.middleware_stack(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/middleware/errors.py", line 186, in call raise exc File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/middleware/errors.py", line 164, in call await self.app(scope, receive, _send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/middleware/exceptions.py", line 65, in call await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/routing.py", line 756, in call await self.middleware_stack(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/routing.py", line 776, in app await route.handle(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/routing.py", line 297, in handle await self.app(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/routing.py", line 77, in app await wrap_app_handling_exceptions(app, request)(scope, receive, send) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/_exception_handler.py", line 64, in wrapped_app raise exc File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app await app(scope, receive, sender) File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/starlette/routing.py", line 72, in app response = await func(request) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/fastapi/routing.py", line 278, in app raw_response = await run_endpoint_function( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/fastapi/routing.py", line 191, in run_endpoint_function return await dependant.call(values) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langserve/server.py", line 530, in invoke return await api_handler.invoke(request) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langserve/api_handler.py", line 833, in invoke output = await invoke_coro ^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5018, in ainvoke return await self.bound.ainvoke( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5018, in ainvoke return await self.bound.ainvoke( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2862, in ainvoke input = await step.ainvoke(input, config) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/branch.py", line 277, in ainvoke output = await runnable.ainvoke( ^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 5018, in ainvoke return await self.bound.ainvoke( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 2860, in ainvoke input = await step.ainvoke(input, config, kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/prompts/base.py", line 203, in ainvoke return await self._acall_with_config( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/runnables/base.py", line 1784, in _acall_with_config output: Output = await asyncio.create_task(coro, context=context) # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/prompts/base.py", line 159, in _aformat_prompt_with_error_handling _inner_input = self._validate_input(inner_input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/var/pyenv/versions/3.11.6/lib/python3.11/site-packages/langchain_core/prompts/base.py", line 145, in _validate_input raise KeyError( KeyError: "Input to ChatPromptTemplate is missing variables {'language'}. Expected: ['history', 'input', 'language'] Received: ['input', 'history']"

Conditions Under Which the Issue Does Not Occur

1. Without Using LangServe:

Running the server-side code (excluding uvicorn.run()) in an IPython shell with the following command does NOT trigger the issue:

chain.invoke(
    {"language": "Italian", "input": "what's my name?"},
    config={"configurable": {"session_id": "1"}},
)

2. Without Using RunnableWithMessageHistory:

Modifying the server-side code as shown below and running it in the playground does NOT trigger the issue:

# Before:
chain = runnable_with_history

# After:
chain = runnable

Conclusion

The issue seems to arise from the combination of RunnableWithMessageHistory and LangServe.

Any assistance or guidance on resolving this issue would be greatly appreciated.

System Info

System Information

OS: Darwin OS Version: Darwin Kernel Version 21.6.0: Thu Jun 8 23:57:12 PDT 2023; root:xnu-8020.240.18.701.6~1/RELEASE_X86_64 Python Version: 3.11.6 (main, Oct 16 2023, 15:57:36) [Clang 14.0.0 (clang-1400.0.29.202)]

Package Information

langchain_core: 0.2.19 langchain: 0.2.7 langchain_community: 0.2.7 langsmith: 0.1.85 langchain_anthropic: 0.1.20 langchain_chroma: 0.1.1 langchain_cli: 0.0.25 langchain_experimental: 0.0.61 langchain_openai: 0.1.16 langchain_text_splitters: 0.2.2 langchainhub: 0.1.20 langchainplus_sdk: 0.0.20 langgraph: 0.1.8 langserve: 0.2.2 pydantic: 2.8.2

langchain-ai / langchain