search

ajndkr / lanarky

The web framework for building LLM microservices
https://lanarky.ajndkr.com/
MIT License
975 stars 74 forks source link

ERROR - Async generation not implemented for this LLM. #49

Closed pakcheera closed 1 year ago

pakcheera commented 1 year ago

​I use the example file "retrieval_qa_w_sources.py" but change the LLM model to GPT4All but It got error that

ERROR - Async generation not implemented for this LLM

Is it not support for that LLM or how I solve it?

ajndkr commented 1 year ago

Hi! I believe it’s a Langchain issue. Since this package uses async methods, likely the agenerate method of the gpt4all llm wrapper is not implemented and hence throws this error.

marcovirgolin commented 1 year ago

I encountered this before and can confirm

ajndkr commented 1 year ago

@pakcheera i suggest adding the async methods first and then trying again.

ajndkr commented 1 year ago

@pakcheera will be closing this for now as there's nothing to do from lanarky end.