search

learningequality / kolibri

Kolibri Learning Platform: the offline app for universal education
https://learningequality.org/kolibri/
MIT License
796 stars 656 forks source link

Migration to 0.16.x does not work on Postgresql #10666

Closed jredrejo closed 1 year ago

jredrejo commented 1 year ago

Observed behavior

After #9292 , the schedulerjobs table is not in use anymore, but there's no migration to remove it from the database. This is causing migration problems in Postgresql, and Kolibri can not start anymore.

Errors and logs

Installed 2 object(s) from 1 fixture(s)
Error: Traceback (most recent call last):
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
psycopg2.errors.DuplicateTable: relation "queue__scheduled_time" already exists

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/utils/cli.py", line 193, in invoke
    initialize(**get_initialize_params())
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/utils/main.py", line 300, in initialize
    update(version, kolibri.__version__)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/utils/main.py", line 366, in update
    run_upgrades(old_version, new_version)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/core/upgrade.py", line 154, in run_upgrades
    filter(filter_upgrade, get_upgrades(app_configs=app_configs))
  File "/datos/le/mio/kolibri-data-portal/kolibri_data_portal/monkey_patch.py", line 28, in my_get_upgrades
    v for v in old_get_upgrades() if v.upgrade.__name__ not in MIGRATIONS_BLACKLIST
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/core/upgrade.py", line 126, in get_upgrades
    version_upgrades += [
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/core/upgrade.py", line 130, in <listcomp>
    if isinstance(upgrade, VersionUpgrade)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/django/utils/functional.py", line 238, in inner
    self._setup()
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/django/utils/functional.py", line 386, in _setup
    self._wrapped = self._setupfunc()
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/core/tasks/main.py", line 18, in __job_storage
    return Storage(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/core/tasks/storage.py", line 81, in __init__
    self.Base.metadata.create_all(self.engine)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/schema.py", line 4930, in create_all
    bind._run_ddl_visitor(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 3228, in _run_ddl_visitor
    conn._run_ddl_visitor(visitorcallable, element, **kwargs)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 2211, in _run_ddl_visitor
    visitorcallable(self.dialect, self, **kwargs).traverse_single(element)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/visitors.py", line 524, in traverse_single
    return meth(obj, **kw)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/ddl.py", line 851, in visit_metadata
    self.traverse_single(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/visitors.py", line 524, in traverse_single
    return meth(obj, **kw)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/ddl.py", line 907, in visit_table
    self.traverse_single(index, create_ok=True)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/visitors.py", line 524, in traverse_single
    return meth(obj, **kw)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/ddl.py", line 938, in visit_index
    self.connection.execute(CreateIndex(index))
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 1380, in execute
    return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
    return connection._execute_ddl(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 1472, in _execute_ddl
    ret = self._execute_context(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 1943, in _execute_context
    self._handle_dbapi_exception(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
    util.raise_(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/util/compat.py", line 210, in raise_
    raise exception
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/base.py", line 1900, in _execute_context
    self.dialect.do_execute(
  File "/datos/le/mio/kolibri-data-portal/venv/lib/python3.10/site-packages/kolibri/dist/sqlalchemy/engine/default.py", line 736, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.ProgrammingError: (psycopg2.errors.DuplicateTable) relation "queue__scheduled_time" already exists

[SQL: CREATE INDEX queue__scheduled_time ON jobs (queue, scheduled_time)]
(Background on this error at: https://sqlalche.me/e/14/f405)

Expected behavior

Starting kolibri should drop current scheduledjobs table (and its indexes) so migrations can happen

User-facing consequences

kolibri won't start after upgrading to 0.16.x if Postgresql is used

Steps to reproduce

rtibbles commented 1 year ago

Is this an issue with the old table not being dropped, or rather with the index being created when it already exists?

jredrejo commented 1 year ago

Is this an issue with the old table not being dropped, or rather with the index being created when it already exists?

The old table is not dropped, thus the index exists when it's being created, causing the error.