search

aws-samples / bedrock-claude-chat

AWS-native chatbot using Bedrock + Claude (+Mistral)
MIT No Attribution
688 stars 232 forks source link

Large amounts of files uploaded but response is SLOW #316

Open dezhouc2 opened 1 month ago

dezhouc2 commented 1 month ago

Describe the bug

Sorry for the confusion, but this is not a bug. I have been using this knowledge database for a while. However, after uploading a large amount of files, the response time has become very slow, sometimes taking more than 10 seconds, which negatively impacts the user experience. In comparison, when a small amount of files are uploaded, the database responds much faster.

I'm wondering why this happens and how we (as users) can achieve quick responses even when a large number of files are uploaded.

Furthermore, the current maximum token limit is 4096. Would it be possible to increase the number of tokens?

Thank you very much!

jeremylatorre commented 1 month ago

Similar to what I experienced https://github.com/aws-samples/bedrock-claude-chat/issues/305

statefb commented 1 month ago

Do you have any public files that can be used to reproduce the issue?

dezhouc2 commented 1 month ago

Similar to what I experienced #305

Have you resolved this issue? Any technique or thought? thank you

dezhouc2 commented 1 month ago

Do you have any public files that can be used to reproduce the issue?

I can provide some files that are public used such as the one in the attachment. Most of my files are internal used which not public (sorry for that). Anyways, for example, once you uploaded 20 files and each has more than 200 pages. Whenever you ask a question or simple conception, the response from knowledge database is extremely SLOW.

PCI-DSS-v3_2_1-ROC-Reporting-Template.pdf

jeremylatorre commented 1 month ago

Similar to what I experienced #305

Have you resolved this issue? Any technique or thought? thank you

No, not yet. But my thought was about the cold start from the lambda in NodeJS CustomHandler. I have to try provisioned concurrency and see if it fixes my problem. For me the problem appears only on the first question, then if you you don't stop asking questions, it's only taking 5s maximum for the next ones

statefb commented 1 month ago

NodeJS CustomHandler

May I ask you which custom handler you are reffering to?

jeremylatorre commented 3 weeks ago

Lambda was a wrong lead, most-likely related to the search in vector database /conversation/related-documents