Docker DB migration necessary after latest update

[ABeltramo]

The latest official Docker image (mozilla/syncserver) has been updated 3 days ago. Since then my installation started failing sync with the following stacktrace:

<class 'sqlalchemy.exc.InternalError'>,
InternalError('(pymysql.err.InternalError) (1054, u"Unknown column \'keys_changed_at\' in \'field list\'")',),
,
[2020-01-23 08:59:21 +0000] [16] [ERROR] Error handling request /token/1.0/sync/1.5,
Traceback (most recent call last):,
  File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 135, in handle,
    self.handle_request(listener, req, client, addr),
  File "/usr/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 176, in handle_request,
    respiter = self.wsgi(environ, resp.start_response),
  File "/usr/local/lib/python2.7/site-packages/pyramid/router.py", line 242, in __call__,
    response = self.invoke_subrequest(request, use_tweens=True),
  File "/usr/local/lib/python2.7/site-packages/pyramid/router.py", line 217, in invoke_subrequest,
    response = handle_request(request),
  File "/usr/local/lib/python2.7/site-packages/tokenserver/tweens.py", line 15, in set_x_timestamp_header_tween,
    response = handler(request),
  File "/usr/local/lib/python2.7/site-packages/syncstorage/tweens.py", line 123, in convert_non_json_responses_tween,
    response = handler(request),
  File "/usr/local/lib/python2.7/site-packages/syncstorage/tweens.py", line 99, in convert_cornice_errors_to_respcodes_tween,
    response = handler(request),
  File "/usr/local/lib/python2.7/site-packages/syncstorage/tweens.py", line 53, in set_default_accept_header_tween,
    return handler(request),
  File "/usr/local/lib/python2.7/site-packages/syncstorage/tweens.py", line 30, in set_x_timestamp_header_tween,
    response = handler(request),
  File "/usr/local/lib/python2.7/site-packages/mozsvc/tweens.py", line 94, in fuzz_backoff_headers_tween,
    response = handler(request),
  File "/usr/local/lib/python2.7/site-packages/mozsvc/tweens.py", line 59, in log_uncaught_exceptions_tween,
    return handler(request),
  File "/usr/local/lib/python2.7/site-packages/mozsvc/tweens.py", line 26, in catch_backend_errors_tween,
    return handler(request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/tweens.py", line 21, in excview_tween,
    response = handler(request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/router.py", line 163, in handle_request,
    response = view_callable(context, request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/config/views.py", line 596, in __call__,
    return view(context, request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/config/views.py", line 329, in attr_view,
    return view(context, request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/config/views.py", line 305, in predicate_wrapper,
    return view(context, request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/config/views.py", line 355, in rendered_view,
    result = view(context, request),
  File "/usr/local/lib/python2.7/site-packages/pyramid/config/views.py", line 501, in _requestonly_view,
    response = view(request),
  File "/usr/local/lib/python2.7/site-packages/cornice/service.py", line 514, in wrapper,
    response = view_(request),
  File "/usr/local/lib/python2.7/site-packages/tokenserver/views.py", line 345, in return_token,
    user = backend.get_user(service, email),
  File "/app/syncserver/staticnode.py", line 135, in get_user,
    res = self._engine.execute(_GET_USER_RECORDS, **params),
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 2166, in execute,
    return connection.execute(statement, *multiparams, **params),
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 988, in execute,
    return meth(self, multiparams, params),
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/sql/elements.py", line 287, in _execute_on_connection,
    return connection._execute_clauseelement(self, multiparams, params),
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1107, in _execute_clauseelement,
    distilled_params,,
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1248, in _execute_context,
    e, statement, parameters, cursor, context,
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1466, in _handle_dbapi_exception,
    util.raise_from_cause(sqlalchemy_exception, exc_info),
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/util/compat.py", line 383, in raise_from_cause,
    reraise(type(exception), exception, tb=exc_tb, cause=cause),
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/base.py", line 1244, in _execute_context,
    cursor, statement, parameters, context,
  File "/usr/local/lib/python2.7/site-packages/sqlalchemy/engine/default.py", line 552, in do_execute,
    cursor.execute(statement, parameters),
  File "/usr/local/lib/python2.7/site-packages/pymysql/cursors.py", line 170, in execute,
    result = self._query(query),
  File "/usr/local/lib/python2.7/site-packages/pymysql/cursors.py", line 328, in _query,
    conn.query(q),
  File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 517, in query,
    self._affected_rows = self._read_query_result(unbuffered=unbuffered),
  File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 732, in _read_query_result,
    result.read(),
  File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 1075, in read,
    first_packet = self.connection._read_packet(),
  File "/usr/local/lib/python2.7/site-packages/pymysql/connections.py", line 684, in _read_packet,
    packet.check_error(),
  File "/usr/local/lib/python2.7/site-packages/pymysql/protocol.py", line 220, in check_error,
    err.raise_mysql_exception(self._data),
  File "/usr/local/lib/python2.7/site-packages/pymysql/err.py", line 109, in raise_mysql_exception,
    raise errorclass(errno, errval),
InternalError: (pymysql.err.InternalError) (1054, u"Unknown column 'keys_changed_at' in 'field list'"),
[SQL: select,
    uid, generation, client_state, created_at, replaced_at,,
    keys_changed_at, node,
from,,
s,
where,
    email = %(email)s,
and,
    service = %(service)s,
order by,
    created_at desc, uid desc,
limit,
    20,
],

I guess it's related to #26 but I don't know how to fix it.

[jrconlin]

Unfortunately, the migration script did not fire. https://github.com/mozilla-services/syncserver/pull/193#commitcomment-36884169 has the fix you should apply to your syncstorage database.

[ABeltramo]

Thanks, fixing the DB solved my issue.