ChainLLM currently supports only the following LLMs for Async API: OpenAI, PromptLayerOpenAI, ChatOpenAI and Anthropic
use_async is a required parameter while adding the langchain_factory decorator though (otherwise you receive ValueError: langchain_factory use_async parameter is required error)
Setting use_async as True throws: Async generation not implemented for this LLM:
Why?
ChainLLM currently supports only the following LLMs for Async API: OpenAI, PromptLayerOpenAI, ChatOpenAI and Anthropic
use_async
is a required parameter while adding thelangchain_factory
decorator though (otherwise you receiveValueError: langchain_factory use_async parameter is required
error)Setting
use_async
asTrue
throws:Async generation not implemented for this LLM
:Setting
use_async
toFalse
fixes the problem: