search

louking / rrwebapp

Race Results Web Application
0 stars 1 forks source link

importresults: lock wait timeout #222

Open louking opened 8 years ago

louking commented 8 years ago
  1. enabled rabbitmq_tracing
  2. restarted rabbitmq-server
  3. attempted importresults

got lock wait timeout when attempt to overwrite results

required touch /var/www/sandbox.scoretility.com/rrwebapp/rrwebapp.wsgi to recover

Message type:       ERROR
Location:           /var/www/sandbox.scoretility.com/rrwebapp/rrwebapp/results.py:1550
Module:             results
Function:           post
Time:               2016-07-16 04:35:15,651

Message:

Traceback (most recent call last):
  File "/var/www/sandbox.scoretility.com/rrwebapp/rrwebapp/results.py", line 1498, in post
    nummrdeleted = ManagedResult.query.filter_by(club_id=club_id,raceid=raceid).delete()
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/orm/query.py", line 2642, in delete
    delete_op.exec_()
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 866, in exec_
    self._do_exec()
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/orm/persistence.py", line 991, in _do_exec
    params=self.query._params)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/orm/session.py", line 978, in execute
    clause, params or {})
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 664, in execute
    return meth(self, multiparams, params)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 282, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 761, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 874, in _execute_context
    context)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1023, in _handle_dbapi_exception
    exc_info
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 185, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 867, in _execute_context
    context)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 388, in do_execute
    cursor.execute(statement, parameters)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/MySQLdb/cursors.py", line 205, in execute
    self.errorhandler(self, exc, value)
  File "/var/www/sandbox.scoretility.com/rrwebapp/lib/python2.7/site-packages/MySQLdb/connections.py", line 36, in defaulterrorhandler
    raise errorclass, errorvalue
  OperationalError: (OperationalError) (1205, 'Lock wait timeout exceeded; try restarting transaction') 'DELETE FROM managedresult WHERE managedresult.club_id = %s AND managedresult.raceid = %s' (2, 187)
louking commented 8 years ago

Upon further review, the lock/wait was probably happening because I restarted the transaction after the first request.

Rather, and this makes more sense, the (existing) apache process thinks it has access to the celery queue and is blocking awaiting the response, as there is no response back to the browser after the http://sandbox.scoretility.com/_importresults/187?force=true request.

So after rabbitmq-server restart, need to restart the apache application, by "touching" the wsgi file.

Not sure if this can be automatically detected and recovered from somehow.