Open VML-Kai opened 7 months ago
Currently using a workaround using FastAPI directly:
@app.post('/categorize_feedback')
async def categorize_feedback(input_data: FeedbackInput) -> AnalysisAttributes:
full_pipeline = prompt | model | parser
output = full_pipeline.invoke(input_data.model_dump())
return output
I think somehow .invoke isn't being called differently in langserve.
Hello @VML-Kai , thanks for the bug report.
Would you be able to tell me:
1) What are your langserve and langchain versions? 2) Which version of pydantic are you using
I'm wondering is whether the automatic type inference is failing when the parser is present and the chain is doing something weird.
If you're able to:
3) Could you share the input and output types of runnable chain? (chain.input_schema.schema() i think or get the jsonschema for those?) 4) Could you manually overload the input/output types:
def with_types(
self,
input_type: Optional[Union[Type[Input], BaseModel]] = None,
output_type: Optional[Union[Type[Output], BaseModel]] = None,
) -> Runnable[Input, Output]:
@VML-Kai I noticed that the URLs that you have a trailing slash
response = requests.post(
"http://localhost:8000/categorize_feedback/invoke/",
json={'input': {'feedback': 'Prices are too high.'}}
)```
Could you remove the trailing slash and try again?
```python
response = requests.post(
"http://localhost:8000/categorize_feedback/invoke",
json={'input': {'feedback': 'Prices are too high.'}}
)
@VML-Kai I was experiencing the same thing, but it works as expected when declaring the output type.
Example on your code:
pipeline = prompt | model | parser add_routes(app, pipeline, path="/categorize_feedback", output_type=AnalysisAttributes)
@danielcj thank you that's helpful! I'll try to reproduce!
I think I'm having this same problem but after some more investigation I'm not sure. Just a simple chain like this
template = "Write a summary of the following article: \"{content}\""
prompt = PromptTemplate(template=template, input_variables=["content"])
add_routes(
app,
prompt | llm,
path="/summary",
)
I'm calling the route from the JS RemoteRunable class. I thought it was just invoke that wasn't working. But now I see both invoke and stream sometimes return a result and sometimes an empty string for the same call. I haven't been able to track down a pattern of why one call works and another doesn't. Just installed langserv yesterday so should be the latest version.
Hello all. I have the same issue I think :
Defining this on server side :
(...)
agent = create_openai_functions_agent(llm, tools, prompt)
agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True)
# Chat history
config={"configurable": {"session_id": "test_session"}}
agent_with_history = RunnableWithMessageHistory(
agent_executor,
lambda session_id: RedisChatMessageHistory(session_id, url=os.getenv('REDIS_URL')),
input_messages_key="input",
history_messages_key="chat_history",
)
app = FastAPI(
title="LangChain Test Server",
version="1.0",
description="A simple API server using LangChain",
)
add_routes(
app,
agent_with_history,
path="/narrator",
)
Then on my "client" side, I try :
def main():
chat = RemoteRunnable("http://localhost:8000/narrator")
session_id = generate_random_string(10)
while True:
human_input = input("You: ") # Get user input
response = chat.invoke({'input': human_input}, {'configurable': {'session_id': session_id}})
print(response)
The response is always an empty {}, while logs in server indicates that everything works fine on server side. (I can see the output displayed in it)
My pip list about this :
langchain 0.1.5
langchain-cli 0.0.21
langchain-community 0.0.17
langchain-core 0.1.18
langchain-openai 0.0.5
langserve 0.0.41
fastapi 0.109.2
Am I missing something ? Thanks in advance !
EDIT :
As @VML-Kai suggested, it works fine with this workaround :
@app.post('/narrator')
async def narrator_chat(input_data : InputData):
output = narrator_chain.invoke({'input': input_data.input}, {'configurable': {'session_id': input_data.session_id}})
return output
I have the same issue when I wrap Agent as a Langserve endpoint with invoke
endpoint
my pip list:
langchain==0.1.14
langchain-cli==0.0.21
langchain-community==0.0.31
langchain-core==0.1.40
langchain-openai==0.1.1
langchain-text-splitters==0.0.1
langserve==0.0.51
langsmith==0.1.40
fastapi==0.110
is this issue solved? i am facing the same problem
i had the same issue, @danielcj 's solution worked for me. Thanks!
I'm building a very simple LangChain application that takes as an input a customer feedback string and categorizes it into the following pydantic class:
Here's how this should work, and it does:
This works very well, all good so far! Let's serve it:
Now comes the strange part, check this out. On the client side, streaming works:
But the regular invoke does not work, it delivers an empty output:
I'm pretty sure the problem is the output parser, as if I remove it from the chain, everything works perfectly fine, the invoke as well.