This pull request enables streaming in Langchain by making the following changes:
Adding the flow.agent.llm_chain.llm.streaming = True line to the main() function.
Adding the cl.LangchainCallbackHandler() callback to the run() method.
These changes will allow the Langchain app to start sending the response to the user as soon as it is available, rather than waiting for the entire response to be returned from the LLM. This can improve the user experience, especially for chatbots and other applications where the user is expecting a response immediately.
Enable streaming in Langchain
Description:
This pull request enables streaming in Langchain by making the following changes:
flow.agent.llm_chain.llm.streaming = True
line to themain()
function.cl.LangchainCallbackHandler()
callback to therun()
method.These changes will allow the Langchain app to start sending the response to the user as soon as it is available, rather than waiting for the entire response to be returned from the LLM. This can improve the user experience, especially for chatbots and other applications where the user is expecting a response immediately.