search

mayooear / gpt4-pdf-chatbot-langchain

GPT4 & LangChain Chatbot for large PDF docs
https://www.youtube.com/watch?v=ih9PBGVVOO4
14.95k stars 3.02k forks source link

Very long "Waiting for server response". From 15 seconds to 3(1) minutes. :( #300

Closed VladislavKatsubo closed 1 year ago

VladislavKatsubo commented 1 year ago

Hello everyone, I've created a simple app using this repo, but with data was in txt files. I used Pinecone Standard plan to store vector data. But i have an issue with request time. Can someone please help me to understand what is the problem with my code?

I have to wait from 15 seconds to 3 minutes(!) for a response when submitting a query to the chat. I figured out by console.time and timeEnd that chain.call() method consumes the majority of the time. Most Next.js hostings offer no more than 60 seconds of Timeout. Can you please suggest how can I figure out what is the problem here and how to fix it?

Here are prints from the terminal: PineconeStore: 1.163ms Model Init: 0.291ms Chain Init: 0.711ms call: 2:58.906 (m:ss.mmm) // Yes, almost three minutes... It was only second or third message to the chat.

I also re-watched part of tutorial where chat.ts code was broken down and noticed that the code there is a bit different. What is the most "up-to-date" version? Github repo or Youtube tutorial? Should recreate chat.ts by Youtube tutorial, maybe?

My Pinecone storage has 400 000 vectors in its index. But as i see on the Pinecone side, maximum request latency is around 100ms.

dosubot[bot] commented 1 year ago

Hi, @VladislavKatsubo! I'm Dosu, and I'm here to help the gpt4-pdf-chatbot-langchain team manage their backlog. I wanted to let you know that we are marking this issue as stale.

From what I understand, you are experiencing long response times when submitting a query to the chat. The chain.call() method seems to be the culprit, causing wait times ranging from 15 seconds to 3 minutes. Currently, there haven't been any updates or comments on this issue.

Before we proceed, we would like to confirm if this issue is still relevant to the latest version of the gpt4-pdf-chatbot-langchain repository. If it is, please let us know by commenting on this issue. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.

Thank you for your understanding, and we look forward to hearing from you soon.