Closed VishwasKukreti closed 11 months ago
What version of chainlit are you using? Can you provide the full error stack trace?
Version is 0.6.402. Two errors pop up during the run. First is 'Exception in ASGI application' and then Error in AsyncLangchainCallbackHandler. The trace for Error in AsyncLangchainCallbackHandler is:
2023-09-03 19:35:35 - Error in AsyncLangchainCallbackHandler.on_llm_start callback: 'NoneType' object is not a mapping
2023-09-03 19:35:35 - 'NoneType' object is not a mapping
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/chainlit/utils.py", line 40, in wrapper
return await user_function(**params_values)
File "/content/cl_f.py", line 170, in main
res = await chain.acall(message, callbacks=[cb])
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 361, in acall
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 355, in acall
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/qa_with_sources/base.py", line 189, in _acall
answer = await self.combine_documents_chain.arun(
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 568, in arun
await self.acall(
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 361, in acall
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 355, in acall
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/combine_documents/base.py", line 121, in _acall
output, extra_return_dict = await self.acombine_docs(
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/combine_documents/stuff.py", line 189, in acombine_docs
return await self.llm_chain.apredict(callbacks=callbacks, **inputs), {}
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/llm.py", line 272, in apredict
return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 361, in acall
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 355, in acall
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/llm.py", line 237, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/llm.py", line 115, in agenerate
return await self.llm.agenerate_prompt(
File "/usr/local/lib/python3.10/dist-packages/langchain/llms/base.py", line 496, in agenerate_prompt
return await self.agenerate(
File "/usr/local/lib/python3.10/dist-packages/langchain/llms/base.py", line 764, in agenerate
run_managers = await asyncio.gather(
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 234, in __step
result = coro.throw(exc)
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 1407, in on_llm_start
await asyncio.gather(*tasks)
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 234, in __step
result = coro.throw(exc)
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 412, in _ahandle_event
await asyncio.gather(
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
result = coro.send(None)
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 397, in _ahandle_event_for_handler
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 367, in _ahandle_event_for_handler
await event(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/chainlit/langchain/callbacks.py", line 528, in on_llm_start
_on_llm_start(self, serialized, prompts, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/chainlit/langchain/callbacks.py", line 355, in _on_llm_start
provider, settings = get_llm_settings(invocation_params, serialized)
File "/usr/local/lib/python3.10/dist-packages/chainlit/langchain/callbacks.py", line 29, in get_llm_settings
merged = {
TypeError: 'NoneType' object is not a mapping
Thank you, what is your version of langchain? can you try updating it?
I am on the latest one 0.6.402.
I think this is your chainlit version. I was asking about langchain :)
Apologies. Lancgchain is the latest one too - 0.0.279
Can you open the langchain/callbacks
in your local chainlit installation and replace the get_llm_settings
with this implementation
def get_llm_settings(invocation_params: Union[Dict, None], serialized: Dict[str, Any]):
if invocation_params is None:
return None, None
provider = invocation_params.pop("_type") # type: str
if provider.startswith("openai"):
model_name = invocation_params.pop("model_name")
invocation_params["model"] = model_name
serialized = serialized or {}
model_kwargs = invocation_params.pop("model_kwargs", {})
merged = {
**invocation_params,
**model_kwargs,
**serialized.get("kwargs", {}),
}
# make sure there is no api key specification
settings = {k: v for k, v in merged.items() if not k.endswith("_api_key")}
return provider, settings
Then, save the file and restart your app. I would like to know if that fixes your issue.
You can find the path of your chainlit installation with pip show chainlit
.
Hey @willydouhard. Thanks for the help. Unfortunately, the error persists.
Unfortunate indeed.
what if you replace it with
def get_llm_settings(invocation_params: Union[Dict, None], serialized: Dict[str, Any]):
return None, None
I want to make sure I scope the problem correctly.
Now it throws cannot unpack non-iterable NoneType object error!
The trace is :
2023-09-04 15:51:44 - Error in AsyncLangchainCallbackHandler.on_llm_start callback: cannot unpack non-iterable NoneType object
2023-09-04 15:51:44 - cannot unpack non-iterable NoneType object
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/chainlit/utils.py", line 40, in wrapper
return await user_function(**params_values)
File "/cl_f.py", line 170, in main
res = await chain.acall(message, callbacks=[cb])
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 361, in acall
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 355, in acall
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/qa_with_sources/base.py", line 189, in _acall
answer = await self.combine_documents_chain.arun(
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 568, in arun
await self.acall(
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 361, in acall
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 355, in acall
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/combine_documents/base.py", line 121, in _acall
output, extra_return_dict = await self.acombine_docs(
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/combine_documents/stuff.py", line 189, in acombine_docs
return await self.llm_chain.apredict(callbacks=callbacks, **inputs), {}
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/llm.py", line 272, in apredict
return (await self.acall(kwargs, callbacks=callbacks))[self.output_key]
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 361, in acall
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/base.py", line 355, in acall
await self._acall(inputs, run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/llm.py", line 237, in _acall
response = await self.agenerate([inputs], run_manager=run_manager)
File "/usr/local/lib/python3.10/dist-packages/langchain/chains/llm.py", line 115, in agenerate
return await self.llm.agenerate_prompt(
File "/usr/local/lib/python3.10/dist-packages/langchain/llms/base.py", line 496, in agenerate_prompt
return await self.agenerate(
File "/usr/local/lib/python3.10/dist-packages/langchain/llms/base.py", line 764, in agenerate
run_managers = await asyncio.gather(
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 234, in __step
result = coro.throw(exc)
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 1407, in on_llm_start
await asyncio.gather(*tasks)
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 234, in __step
result = coro.throw(exc)
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 412, in _ahandle_event
await asyncio.gather(
File "/usr/lib/python3.10/asyncio/tasks.py", line 304, in __wakeup
future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 232, in __step
result = coro.send(None)
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 397, in _ahandle_event_for_handler
raise e
File "/usr/local/lib/python3.10/dist-packages/langchain/callbacks/manager.py", line 367, in _ahandle_event_for_handler
await event(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/chainlit/langchain/callbacks.py", line 509, in on_llm_start
_on_llm_start(self, serialized, prompts, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/chainlit/langchain/callbacks.py", line 336, in _on_llm_start
provider, settings = get_llm_settings(invocation_params, serialized)
TypeError: cannot unpack non-iterable NoneType object
Are you sure the indentation is correct? I cannot reproduce at all, even the last error.
Hey @willydouhard, by 'cannot reproduce' do you mean the app works or it does not work at all? I find the indentation to be correct. I repeat every instruction twice before getting back to you. Would you be more comfortable if I share the code with you?
I mean my app works. The code would be helpful! Information about your setup would also help (OS, python version). Thank you for taking the time!
Im getting a similar error..
2023-09-06 18:27:08 - Error in AsyncLangchainCallbackHandler.on_chat_model_start callback: '_type'
2023-09-06 18:27:08 - Error in AsyncLangchainCallbackHandler.on_chat_model_start callback: '_type'
2023-09-06 18:27:08 - Error in AsyncLangchainCallbackHandler.on_chat_model_start callback: '_type'
2023-09-06 18:27:08 - '_type'
Traceback (most recent call last):
File "/home/vscode/.local/lib/python3.9/site-packages/chainlit/utils.py", line 40, in wrapper
return await user_function(**params_values)
File "app.py", line 94, in main
res = await chain.acall(inputs={"question": message}, callbacks=[cb])
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/base.py", line 349, in acall
raise e
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/base.py", line 343, in acall
await self._acall(inputs, run_manager=run_manager)
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/conversational_retrieval/base.py", line 190, in _acall
answer = await self.combine_docs_chain.arun(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/base.py", line 556, in arun
await self.acall(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/base.py", line 349, in acall
raise e
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/base.py", line 343, in acall
await self._acall(inputs, run_manager=run_manager)
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/combine_documents/base.py", line 122, in _acall
output, extra_return_dict = await self.acombine_docs(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/combine_documents/map_reduce.py", line 241, in acombine_docs
map_results = await self.llm_chain.aapply(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/llm.py", line 206, in aapply
raise e
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/llm.py", line 203, in aapply
response = await self.agenerate(input_list, run_manager=run_manager)
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chains/llm.py", line 116, in agenerate
return await self.llm.agenerate_prompt(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chat_models/base.py", line 423, in agenerate_prompt
return await self.agenerate(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/chat_models/base.py", line 348, in agenerate
run_managers = await callback_manager.on_chat_model_start(
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/callbacks/manager.py", line 1426, in on_chat_model_start
await asyncio.gather(*tasks)
File "/usr/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
future.result()
File "/usr/lib/python3.9/asyncio/tasks.py", line 258, in __step
result = coro.throw(exc)
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/callbacks/manager.py", line 383, in _ahandle_event
await asyncio.gather(
File "/usr/lib/python3.9/asyncio/tasks.py", line 328, in __wakeup
future.result()
File "/usr/lib/python3.9/asyncio/tasks.py", line 256, in __step
result = coro.send(None)
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/callbacks/manager.py", line 368, in _ahandle_event_for_handler
raise e
File "/home/vscode/.local/lib/python3.9/site-packages/langchain/callbacks/manager.py", line 338, in _ahandle_event_for_handler
await event(*args, **kwargs)
File "/home/vscode/.local/lib/python3.9/site-packages/chainlit/langchain/callbacks.py", line 536, in on_chat_model_start
_on_chat_model_start(self, serialized, messages, **kwargs)
File "/home/vscode/.local/lib/python3.9/site-packages/chainlit/langchain/callbacks.py", line 284, in _on_chat_model_start
provider, settings = get_llm_settings(invocation_params, serialized)
File "/home/vscode/.local/lib/python3.9/site-packages/chainlit/langchain/callbacks.py", line 21, in get_llm_settings
provider = invocation_params.pop("_type") # type: str
KeyError: '_type'
This one I understand, the easy fix is to add a default argument none to the pop function. Will release a fix https://github.com/Chainlit/chainlit/pull/368/files
Trying to follow the document QA example on local dataset with the exception of using llama-2 instead of OpenAI. I am stumped by the error in AsyncLangChainCallbackHandler. Whatever I have tried has been unsuccessful. How to overcome the error?
My code is: