calpoly-csai / api

Official API for the NIMBUS Voice Assistant accessible via HTTP REST protocol.
https://nimbus.api.calpolycsai.com/
GNU General Public License v3.0
9 stars 4 forks source link

Broken MySQL Sessions #197

Closed snekiam closed 3 years ago

snekiam commented 3 years ago

Describe the bug Getting the following response when asking any question:

{
    "ErrorLog": "InvalidRequestError"
}

Caused by a failed flush, then the session needs to be recreated. It's strange that this is still happening, since I thought it was fixed.

Traceback

Traceback (most recent call last):
   File "/nimbus/flask_api.py", line 169, in handle_question
     response = {"answer": nimbus.answer_question(question)}
   File "/nimbus/nimbus.py", line 41, in answer_question
     answer = qa.answer(ans_dict)
   File "/nimbus/QA.py", line 52, in answer
     db_data = self.db_query(extracted_vars, self.db)
   File "/nimbus/QA.py", line 231, in _chain_db_access
     for key, val in fn(extracted_vars, db).items():
   File "/nimbus/QA.py", line 94, in _get_property
     value = db.get_property_from_entity(
   File "/nimbus/database_wrapper.py", line 542, in get_property_from_entity
     props = self._get_property_from_entity(prop, entity, identifier, tag_column_map)
   File "/nimbus/database_wrapper.py", line 515, in _get_property_from_entity
     for row in query_obj.all():
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 3233, in all
     return list(self)
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/query.py", line 3388, in __iter__
     self.session._autoflush()
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1586, in _autoflush
     self.flush()
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 2479, in flush
     self._flush(objects)
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 2571, in _flush
     flush_context.transaction = transaction = self.begin(
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 947, in begin
     self.transaction = self.transaction._begin(nested=nested)
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 316, in _begin
     self._assert_active()
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 288, in _assert_active
     raise sa_exc.InvalidRequestError(
 sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (mysql.connector.errors.OperationalError) MySQL Connection not available.
 [SQL: INSERT INTO `ErrorLog` (question, stacktrace, timestamp) VALUES (%(question)s, %(stacktrace)s, %(timestamp)s)]
 [parameters: [{'timestamp': datetime.datetime(2020, 9, 8, 2, 13, 57, 52278), 'stacktrace': 'Traceback (most recent call last):\n  File "/usr/local/lib/python3.8/di ... (664 characters truncated) ... Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.\n', 'question': None}]]
 (Background on this error at: http://sqlalche.me/e/e3q8) (Background on this error at: http://sqlalche.me/e/7s2a)

 During handling of the above exception, another exception occurred:

 Traceback (most recent call last):
   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1949, in full_dispatch_request
     rv = self.dispatch_request()
   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1935, in dispatch_request
     return self.view_functions[rule.endpoint](**req.view_args)
   File "/nimbus/flask_api.py", line 179, in handle_question
     log_error(e, question)
   File "/nimbus/flask_api.py", line 112, in log_error
     db.insert_entity(ErrorLog, error_entry)
   File "/nimbus/database_wrapper.py", line 672, in insert_entity
     self.session.commit()
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1036, in commit
     self.transaction.commit()
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 501, in commit
     self._assert_active(prepared_ok=True)
   File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 288, in _assert_active
     raise sa_exc.InvalidRequestError(
 sqlalchemy.exc.InvalidRequestError: This Session's transaction has been rolled back due to a previous exception during flush. To begin a new transaction with this Session, first issue Session.rollback(). Original exception was: (mysql.connector.errors.OperationalError) MySQL Connection not available.
 [SQL: INSERT INTO `ErrorLog` (question, stacktrace, timestamp) VALUES (%(question)s, %(stacktrace)s, %(timestamp)s)]
 [parameters: [{'timestamp': datetime.datetime(2020, 9, 8, 2, 13, 57, 52278), 'stacktrace': 'Traceback (most recent call last):\n  File "/usr/local/lib/python3.8/di ... (664 characters truncated) ... Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.\n', 'question': None}]]
 (Background on this error at: http://sqlalche.me/e/e3q8) (Background on this error at: http://sqlalche.me/e/7s2a)
snekiam commented 3 years ago

I just restarted the gcp instance to bring it back online. I'll be debugging this when I've got a chance.