When using gpt4-turbo on azure. The partial stream takes several minutes (azure endpoint issue).
Because of this :point_up_2: it seems that the connexion to postgresql is closed when trying to insert the answer :
05/30/2024 09:08:24 AM process_message.py 511 : (psycopg2.OperationalError) SSL connection has been closed unexpectedly
[SQL: INSERT INTO chat_message (chat_session_id, parent_message, latest_child_message, message, rephrased_query, prompt_id, token_count, message_type, citations, files, error) VALUES (%(chat_session_id)s, %(parent_message)s, %(latest_child_message)s, %(message)s, %(rephrased_query)s, %(prompt_id)s, %(token_count)s, %(message_type)s, %(citations)s, %(files)s, %(error)s) RETURNING chat_message.id, chat_message.time_sent]
[parameters: {'chat_session_id': 439, 'parent_message': 753, 'latest_child_message': None, 'message': "It appears you've gathered detailed feedback from a survey assessing satisfaction rates on various search terms related to online shopping for home a ... (14 characters truncated) ... lies, tools, and more. The rates range from 1 (lowest satisfaction) to 5 (highest satisfaction). \n\nHere's a summary of key observations:\n1. **High", 'rephrased_query': None, 'prompt_id': 167, 'token_count': 62, 'message_type': 'ASSISTANT', 'citations': 'null', 'files': 'null', 'error': None}]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1968, in _exec_single_context
self.dialect.do_execute(
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 920, in do_execute
cursor.execute(statement, parameters)
psycopg2.OperationalError: SSL connection has been closed unexpectedly
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/app/danswer/chat/process_message.py", line 493, in stream_chat_message_objects
gen_ai_response_message = partial_response(
^^^^^^^^^^^^^^^^^
File "/app/danswer/db/chat.py", line 292, in create_new_chat_message
db_session.flush()
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4154, in flush
self._flush(objects)
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4290, in _flush
with util.safe_reraise():
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/util/langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 4251, in _flush
flush_context.execute()
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 467, in execute
rec.execute(self)
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/unitofwork.py", line 644, in execute
util.preloaded.orm_persistence.save_obj(
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/persistence.py", line 93, in save_obj
_emit_insert_statements(
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/orm/persistence.py", line 1223, in _emit_insert_statements
result = connection.execute(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1413, in execute
return meth(
^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 483, in _execute_on_connection
return connection._execute_clauseelement(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1637, in _execute_clauseelement
ret = self._execute_context(
^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1846, in _execute_context
return self._exec_single_context(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1987, in _exec_single_context
self._handle_dbapi_exception(
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2344, in _handle_dbapi_exception
raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1968, in _exec_single_context
self.dialect.do_execute(
File "/usr/local/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 920, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL connection has been closed unexpectedly
[SQL: INSERT INTO chat_message (chat_session_id, parent_message, latest_child_message, message, rephrased_query, prompt_id, token_count, message_type, citations, files, error) VALUES (%(chat_session_id)s, %(parent_message)s, %(latest_child_message)s, %(message)s, %(rephrased_query)s, %(prompt_id)s, %(token_count)s, %(message_type)s, %(citations)s, %(files)s, %(error)s) RETURNING chat_message.id, chat_message.time_sent]
[parameters: {'chat_session_id': 439, 'parent_message': 753, 'latest_child_message': None, 'message': "It appears you've gathered detailed feedback from a survey assessing satisfaction rates on various search terms related to online shopping for home a ... (14 characters truncated) ... lies, tools, and more. The rates range from 1 (lowest satisfaction) to 5 (highest satisfaction). \n\nHere's a summary of key observations:\n1. **High", 'rephrased_query': None, 'prompt_id': 167, 'token_count': 62, 'message_type': 'ASSISTANT', 'citations': 'null', 'files': 'null', 'error': None}]
(Background on this error at: https://sqlalche.me/e/20/e3q8)
05/30/2024 09:08:24 AM timing.py 74 : stream_chat_message took 104.46927809715271 seconds
05/30/2024 09:08:24 AM chat_backend.py 174 : Received rename request for chat session: 439
05/30/2024 09:08:25 AM chat_session_naming.py 47 : New Session Name: Search Relevancy Survey Results
INFO: 127.0.0.6:45691 - "PUT /chat/rename-chat-session HTTP/1.1" 200 OK
INFO: 127.0.0.1:46814 - "GET /auth/type HTTP/1.1" 200 OK
INFO: 127.0.0.1:46822 - "GET /manage/me HTTP/1.1" 200 OK
INFO: 127.0.0.1:46868 - "GET /chat/get-user-chat-sessions HTTP/1.1" 200 OK
The UI shows the following error Failed to parse LLM outputand deletes the LLM response.
For the moment one quick fix could be done on the backend/danswer/chat/process_message.py file that only queries the database with a build in function that you implemented :
for packet in answer.processed_streamed_output:
# Keeping alive session for long running LLM calls
_ = get_db_current_time(db_session)
On version
v0.3.75When using
gpt4-turboon azure. The partial stream takes several minutes (azure endpoint issue).Because of this :point_up_2: it seems that the connexion to postgresql is closed when trying to insert the answer :
The UI shows the following error
Failed to parse LLM outputand deletes the LLM response.For the moment one quick fix could be done on the
backend/danswer/chat/process_message.pyfile that only queries the database with a build in function that you implemented :If that suits you well we can create a PR.