search

brainlid / langchain

Elixir implementation of an AI focused LangChain style framework.
https://hexdocs.pm/langchain/
Other
571 stars 68 forks source link

Multiple tool calls: Frequent timeouts #129

Closed cpursley closed 3 months ago

cpursley commented 3 months ago

I'm experiencing this frequency: 

** (exit) exited in: Task.await_many([%Task{mfa: {:erlang, :apply, 2}, owner: #PID<0.27977.0>, pid: #PID<0.29053.0>, ref: #Reference<0.0.3581059.2102371376.702611457.169984>}], 5000)
    ** (EXIT) time out
    (elixir 1.15.7) lib/task.ex:969: Task.await_many/5
    (elixir 1.15.7) lib/task.ex:953: Task.await_many/2
    (langchain 0.2.0) lib/chains/llm_chain.ex:456: LangChain.Chains.LLMChain.execute_tool_calls/2
    (langchain 0.2.0) lib/chains/llm_chain.ex:189: LangChain.Chains.LLMChain.run_while_needs_response/1

Can we make the Task.await_many timeout configurable (either via config or pass as an optional arg in the run call)? Glad to submit a PR.

brainlid commented 3 months ago

Ah, yes, I can see it might. Some tools may do RAG lookups, query web pages, etc. A configurable timeout makes sense. What were you thinking? It could be an option on the run. It could be a library config option. And regardless, it should probably default to a higher timeout that the default 5 seconds!

cpursley commented 3 months ago

I'm leaning towards opt in run

brainlid commented 3 months ago

In v0.3.0-rc.0, I added an explicit timeout of 2 minutes. That should get you past the immediate issue, but I agree that timeout control would be nice.