Timeout on longer generations: Part 2

0ptim / JellyChat

🪼 AI chatbot for the DeFiChain ecosystem.

https://defichainwiki.com/jellychat

MIT License

11 stars 2 forks source link

Timeout on longer generations: Part 2 #48

Closed 0ptim closed 1 year ago

0ptim commented 1 year ago

Because we still want to support a REST endpoint, we still need to find a solution to this problem.

Read more details in Part 1: #47

0ptim commented 1 year ago

Since llm text generation got faster by a big leap and will probably improve even more in the near future, I will not fix this until it becomes a more problematic issue.