search

microsoft / sample-app-aoai-chatGPT

Sample code for a simple web chat experience through Azure OpenAI, including Azure OpenAI On Your Data.
MIT License
1.44k stars 2.15k forks source link

App "Generating" in the background at start when history is enabled, no issue if disabled #823

Open Francois2024 opened 2 months ago

Francois2024 commented 2 months ago

Describe the bug If I enable the history option, the app tries to generate content when the url is loaded when starting the app. This prevents the user from using the chatbot for 5-10 seconds. No content is generated in the end but there is a process running in the background which shows the app as being in content generation mode.

To Reproduce Steps to reproduce the behavior:

  1. Load your app with history option enabled
  2. "Stop Generating" is active for 5-10 sec (eventhough no content has been entered in the chat window)
  3. Wait 5-10 sec and then user can start prompting the chatbot

Expected behavior Same behaviour as when history is not enabled. No content generation attempt at start.

Screenshots If applicable, add screenshots to help explain your problem.

Configuration: Please provide the following

Additional context Add any other context about the problem here.

neissljanik commented 1 month ago

There's no content actually being generated. In the first few seconds the app connects to the history database and reads your chats from there. Until these requests have completed, the application shows the "generating" hint. That's also the reason this behavior only occurs when history is enabled.

Francois2024 commented 1 month ago

Thanks for the explanation. Is there a way to disable this behavior so it does not read the chats history? This is not useful for us and it creates a 5-10 seconds loading delay before the user can query the app.

neissljanik commented 1 month ago

Sort of, but not without some programming knowledge. We implemented lazy loading for chat messages in our fork, which helps to reduce the number of requests sent initially and therefore cuts down on loading time. However, if you want to keep the history feature, there's no sensible way around establishing a connection to the database initially. You can't save messages to the DB if there's no connection to the DB.

Francois2024 commented 1 month ago

Ok got it thanks for your help. We will look into it.

Francois2024 commented 1 month ago

We removed the history feature for now until we find a solution.

neissljanik commented 1 month ago

FWIW, our fork currently has 4.5s of loading time, but we're doing some additional requests as well, so implementing lazy loading for the chat messages would probably reduce the initial wait to 3-4s. Keep in mind that this is a very rough estimation influenced by a boatload of factors.

Francois2024 commented 1 month ago

Ok! Thanks. This repo is by the way very useful as a starting point.