statistics.py: sqlalchemy.exc.StatementError: (builtins.AssertionError)

[kynikos]

Hi, sorry to ruin your Saturday morning, GitHub notifications are seldom good news haha :P Fact is I tried to run statistics.py today, and the first time I got:

Traceback (most recent call last):
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1204, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 873, in _init_compiled
    param = dict(
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 876, in <genexpr>
    processors[key](compiled_params[key])
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1229, in process
    return process_param(value, dialect)
  File "/home/dario/dev/arch/wiki-scripts/ws/db/sql_types.py", line 63, in process_bind_param
    assert isinstance(value, datetime.datetime), value
AssertionError: 2021-04-16T12:29:46Z

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 311, in <module>
    sys.exit(statistics.run())
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 98, in run
    self.db.sync_with_api(self.api)
  File "/home/dario/dev/arch/wiki-scripts/ws/db/database.py", line 147, in sync_with_api
    grabbers.synchronize(self, api, with_content=with_content, check_needs_update=check_needs_update)
  File "/home/dario/dev/arch/wiki-scripts/ws/db/grabbers/__init__.py", line 39, in synchronize
    GrabberRevisions(api, db, with_content=with_content).update()
  File "/home/dario/dev/arch/wiki-scripts/ws/db/grabbers/GrabberBase.py", line 129, in update
    self._execute(gen, sync_timestamp)
  File "/home/dario/dev/arch/wiki-scripts/ws/db/grabbers/GrabberBase.py", line 143, in _execute
    dfe.execute(item)
  File "/home/dario/dev/arch/wiki-scripts/ws/db/execution.py", line 74, in __exit__
    self.execute_deferred()
  File "/home/dario/dev/arch/wiki-scripts/ws/db/execution.py", line 65, in execute_deferred
    self.conn.execute(statement, self.stmt_queues[statement])
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1011, in execute
    return meth(self, multiparams, params)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/elements.py", line 298, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1124, in _execute_clauseelement
    ret = self._execute_context(
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1206, in _execute_context
    self._handle_dbapi_exception(
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1510, in _handle_dbapi_exception
    util.raise_(
  File "/usr/lib/python3.9/site-packages/sqlalchemy/util/compat.py", line 182, in raise_
    raise exception
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/base.py", line 1204, in _execute_context
    context = constructor(dialect, self, conn, *args)
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 873, in _init_compiled
    param = dict(
  File "/usr/lib/python3.9/site-packages/sqlalchemy/engine/default.py", line 876, in <genexpr>
    processors[key](compiled_params[key])
  File "/usr/lib/python3.9/site-packages/sqlalchemy/sql/type_api.py", line 1229, in process
    return process_param(value, dialect)
  File "/home/dario/dev/arch/wiki-scripts/ws/db/sql_types.py", line 63, in process_bind_param
    assert isinstance(value, datetime.datetime), value
sqlalchemy.exc.StatementError: (builtins.AssertionError) 2021-04-16T12:29:46Z
[SQL: UPDATE revision SET rev_page=(SELECT page.page_id
FROM page
WHERE page.page_namespace = %(b_dest_ns)s AND page.page_title = %(b_dest_title)s) WHERE revision.rev_page = %(b_src_page_id)s AND revision.rev_timestamp <= %(b_mergepoint)s]
[parameters: [{'b_src_page_id': 28534, 'b_dest_ns': 0, 'b_dest_title': 'Dell Inspiron 7375', 'b_mergepoint': '2021-04-16T12:29:46Z'}]]

All the following attempts only output this instead:

INFO     No new changes since the last database synchronization.
Traceback (most recent call last):
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 311, in <module>
    sys.exit(statistics.run())
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 109, in run
    self._compose_page()
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 119, in _compose_page
    userstats.update()
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 234, in update
    rows = self._compose_rows()
  File "/home/dario/dev/arch/wiki-scripts/statistics.py", line 273, in _compose_rows
    longest_streak, current_streak = self.modules.get_streaks(name)
  File "/home/dario/dev/arch/wiki-scripts/ws/statistics/UserStatsModules.py", line 81, in get_streaks
    streak_groups = itertools.groupby(self.revisions_groups[user], key=_streak)
KeyError: 'ADSWT518'

[lahwaacz]

Doh! I fixed it in https://github.com/lahwaacz/wiki-scripts/commit/78eff46311c112fe7725bb17ea5704d6b031320b a few days ago actually, but forgot to push the commit... :shrug:

[kynikos]

It works, happy days!