search

cgwire / zou

Zou is the Kitsu API. It allows you to store and manage your production data
https://zou.cg-wire.com
GNU Affero General Public License v3.0
166 stars 103 forks source link

upgrade-db fails after upgrading from 0.17.63 to any newer version #784

Closed flinfo closed 6 months ago

flinfo commented 6 months ago

Attempting to upgrade to latest versions of Zou and Kitsu. Started upgrade from 0.17.46 > 0.17.63. Upgrade of zou, then upgrade-db both complete successfully. 0.17.63 > 0.18.0 (also tried 0.18.2) completed successfully, but upgrade-db fails with below traceback:

Zou version: 0.18.0 Zou installation type: self-hosted

Describe the bug upgrade-db fails after upgrading from 0.17.63 to any newer version.

Expected behavior upgrade-db is expected to complete successfully

traceback:

 File "/opt/zou/zouenv/bin/zou", line 8, in <module>
    sys.exit(cli())
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/zou/cli.py", line 98, in upgrade_db
    flask_migrate.upgrade(directory=migrations_path)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 111, in wrapped
    f(*args, **kwargs)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 200, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/command.py", line 378, in upgrade
    script.run_env()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/script/base.py", line 576, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
    module = load_module_py(module_id, path)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/zou/zouenv/lib/python3.8/site-packages/zou/migrations/env.py", line 24, in <module>
    config.set_main_option(
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/config.py", line 251, in set_main_option
    self.set_section_option(self.config_ini_section, name, value)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/config.py", line 278, in set_section_option
    self.file_config.set(section, name, value)
  File "/usr/lib/python3.8/configparser.py", line 1201, in set
    super().set(section, option, value)
  File "/usr/lib/python3.8/configparser.py", line 894, in set
    value = self._interpolation.before_set(self, section, option,
  File "/usr/lib/python3.8/configparser.py", line 402, in before_set
    raise ValueError("invalid interpolation syntax in %r at "
ValueError: invalid interpolation syntax in 'postgresql+psycopg://postgres:%5Buse%2Cr@localhost:5432/zoudb3' at position 30
flinfo commented 6 months ago

the process I am using:

sudo systemctl stop zou zou-events
sudo /opt/zou/zouenv/bin/pip3 install --upgrade zou==0.18.0
sudo -u zou DB_PASSWORD=passwordgoeshere DB_DATABASE=dbnamegoeshere /opt/zou/zouenv/bin/zou upgrade-db
sudo systemctl restart postgresql zou zou-events
flinfo commented 6 months ago

trying on a clone:

Traceback (most recent call last):
  File "/opt/zou/zouenv/bin/zou", line 8, in <module>
    sys.exit(cli())
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1688, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 1434, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/click/core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/zou/cli.py", line 104, in upgrade_db
    flask_migrate.upgrade(directory=migrations_path)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 111, in wrapped
    f(*args, **kwargs)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/flask_migrate/__init__.py", line 200, in upgrade
    command.upgrade(config, revision, sql=sql, tag=tag)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/command.py", line 399, in upgrade
    script.run_env()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/script/base.py", line 578, in run_env
    util.load_python_file(self.dir, "env.py")
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
    module = load_module_py(module_id, path)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
    spec.loader.exec_module(module)  # type: ignore
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/zou/zouenv/lib/python3.8/site-packages/zou/migrations/env.py", line 108, in <module>
    run_migrations_online()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/zou/migrations/env.py", line 74, in run_migrations_online
    connection = engine.connect()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3279, in connect
    return self._connection_cls(self)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 147, in __init__
    Connection._handle_dbapi_exception_noconnection(
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 2443, in _handle_dbapi_exception_noconnection
    raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 145, in __init__
    self._dbapi_connection = engine.raw_connection()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/base.py", line 3303, in raw_connection
    return self.pool.connect()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 449, in connect
    return _ConnectionFairy._checkout(self)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 1263, in _checkout
    fairy = _ConnectionRecord.checkout(pool)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 712, in checkout
    rec = pool._do_get()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/impl.py", line 308, in _do_get
    return self._create_connection()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 390, in _create_connection
    return _ConnectionRecord(self)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 674, in __init__
    self.__connect()
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 901, in __connect
    pool.logger.debug("Error on connect(): %s", e)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 146, in __exit__
    raise exc_value.with_traceback(exc_tb)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/pool/base.py", line 896, in __connect
    self.dbapi_connection = connection = pool._invoke_creator(self)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/create.py", line 643, in connect
    return dialect.connect(*cargs, **cparams)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/sqlalchemy/engine/default.py", line 616, in connect
    return self.loaded_dbapi.connect(*cargs, **cparams)
  File "/opt/zou/zouenv/lib/python3.8/site-packages/psycopg/connection.py", line 748, in connect
    raise last_ex.with_traceback(None)
sqlalchemy.exc.OperationalError: (psycopg.OperationalError) connection failed: Connection refused
        Is the server running on that host and accepting TCP/IP connections?
(Background on this error at: https://sqlalche.me/e/20/e3q8)

Any ideas?

flinfo commented 6 months ago

For some reason, in the upgrade process postgresql must have kept stopping or I was stopping it? It worked as soon as I started it again. Thankfully I have multiple instances running and when I tried on another instance, it gave me the clue: "connection failed: Connection refused. Is the host running and accepting tcp/ip connections".

so just had to sudo systemctl start postgresql

Learning :)