search

TomBursch / kitchenowl

KitchenOwl is a self-hosted grocery list and recipe manager. The backend is made with Flask and the frontend with Flutter. Easily add items to your shopping list before you go shopping. You can also create recipes and add items based on what you want to cook.
https://kitchenowl.org/
GNU Affero General Public License v3.0
1.2k stars 66 forks source link

Bug: new instance migration issue #252

Closed from-nibly closed 11 months ago

from-nibly commented 11 months ago

Is this urgent?

Yes

What parts are affected

Backend

What are the Server/Client versions of KitchenOwl

backend v77

What's the problem 🤔

when trying to launch a new instance that uses postgresql It starts the migrations but dies half way through with the below error.

I'm not sure if it's running a transaction for all the migrations but when I connect to the database there are 0 tables added.

This is preventing me from starting a new instance. I've run out of time for today but my next attempt would be to try running older versions of kitchenowl

very excited to get this working, thank you for making this. let me know if there is more info you need.

Share your logs

kitchenowl-f9b4d7776-qztzm kitchenowl INFO  [alembic.runtime.migration] Running upgrade 3647c9eb1881 -> ade9ad0be1a5, empty message
kitchenowl-f9b4d7776-qztzm kitchenowl Traceback (most recent call last):
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/bin/flask", line 8, in <module>
kitchenowl-f9b4d7776-qztzm kitchenowl     sys.exit(main())
kitchenowl-f9b4d7776-qztzm kitchenowl              ^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/flask/cli.py", line 1064, in main
kitchenowl-f9b4d7776-qztzm kitchenowl     cli.main()
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/core.py", line 1078, in main
kitchenowl-f9b4d7776-qztzm kitchenowl     rv = self.invoke(ctx)
kitchenowl-f9b4d7776-qztzm kitchenowl          ^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
kitchenowl-f9b4d7776-qztzm kitchenowl     return _process_result(sub_ctx.command.invoke(sub_ctx))
kitchenowl-f9b4d7776-qztzm kitchenowl                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/core.py", line 1688, in invoke
kitchenowl-f9b4d7776-qztzm kitchenowl     return _process_result(sub_ctx.command.invoke(sub_ctx))
kitchenowl-f9b4d7776-qztzm kitchenowl                            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/core.py", line 1434, in invoke
kitchenowl-f9b4d7776-qztzm kitchenowl     return ctx.invoke(self.callback, **ctx.params)
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
kitchenowl-f9b4d7776-qztzm kitchenowl     return __callback(*args, **kwargs)
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/decorators.py", line 33, in new_func
kitchenowl-f9b4d7776-qztzm kitchenowl     return f(get_current_context(), *args, **kwargs)
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/flask/cli.py", line 358, in decorator
kitchenowl-f9b4d7776-qztzm kitchenowl     return __ctx.invoke(f, *args, **kwargs)
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/click/core.py", line 783, in invoke
kitchenowl-f9b4d7776-qztzm kitchenowl     return __callback(*args, **kwargs)
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/flask_migrate/cli.py", line 150, in upgrade
kitchenowl-f9b4d7776-qztzm kitchenowl     _upgrade(directory, revision, sql, tag, x_arg)
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/flask_migrate/__init__.py", line 111, in wrapped
kitchenowl-f9b4d7776-qztzm kitchenowl     f(*args, **kwargs)
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/flask_migrate/__init__.py", line 200, in upgrade
kitchenowl-f9b4d7776-qztzm kitchenowl     command.upgrade(config, revision, sql=sql, tag=tag)
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/alembic/command.py", line 399, in upgrade
kitchenowl-f9b4d7776-qztzm kitchenowl     script.run_env()
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/alembic/script/base.py", line 578, in run_env
kitchenowl-f9b4d7776-qztzm kitchenowl     util.load_python_file(self.dir, "env.py")
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 93, in load_python_file
kitchenowl-f9b4d7776-qztzm kitchenowl     module = load_module_py(module_id, path)
kitchenowl-f9b4d7776-qztzm kitchenowl              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/alembic/util/pyfiles.py", line 109, in load_module_py
kitchenowl-f9b4d7776-qztzm kitchenowl     spec.loader.exec_module(module)  # type: ignore
kitchenowl-f9b4d7776-qztzm kitchenowl     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "<frozen importlib._bootstrap_external>", line 940, in exec_module
kitchenowl-f9b4d7776-qztzm kitchenowl   File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/usr/src/kitchenowl/migrations/env.py", line 110, in <module>
kitchenowl-f9b4d7776-qztzm kitchenowl     run_migrations_online()
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/usr/src/kitchenowl/migrations/env.py", line 104, in run_migrations_online
kitchenowl-f9b4d7776-qztzm kitchenowl     context.run_migrations()
kitchenowl-f9b4d7776-qztzm kitchenowl   File "<string>", line 8, in run_migrations
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/alembic/runtime/environment.py", line 937, in run_migrations
kitchenowl-f9b4d7776-qztzm kitchenowl     self.get_context().run_migrations(**kw)
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/alembic/runtime/migration.py", line 624, in run_migrations
kitchenowl-f9b4d7776-qztzm kitchenowl     step.migration_fn(**kw)
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/usr/src/kitchenowl/migrations/versions/ade9ad0be1a5_.py", line 36, in upgrade
kitchenowl-f9b4d7776-qztzm kitchenowl     if not any(c['name'] == 'blur_hash' for c in inspector.get_columns('file')):
kitchenowl-f9b4d7776-qztzm kitchenowl                                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 859, in get_columns
kitchenowl-f9b4d7776-qztzm kitchenowl     col_defs = self.dialect.get_columns(
kitchenowl-f9b4d7776-qztzm kitchenowl                ^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "<string>", line 2, in get_columns
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/reflection.py", line 97, in cache
kitchenowl-f9b4d7776-qztzm kitchenowl     ret = fn(self, con, *args, **kw)
kitchenowl-f9b4d7776-qztzm kitchenowl           ^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/base.py", line 3481, in get_columns
kitchenowl-f9b4d7776-qztzm kitchenowl     return self._value_or_raise(data, table_name, schema)
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/dialects/postgresql/base.py", line 3449, in _value_or_raise
kitchenowl-f9b4d7776-qztzm kitchenowl     raise exc.NoSuchTableError(
kitchenowl-f9b4d7776-qztzm kitchenowl sqlalchemy.exc.NoSuchTableError: file
kitchenowl-f9b4d7776-qztzm kitchenowl Traceback (most recent call last):
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
kitchenowl-f9b4d7776-qztzm kitchenowl     self.dialect.do_execute(
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
kitchenowl-f9b4d7776-qztzm kitchenowl     cursor.execute(statement, parameters)
kitchenowl-f9b4d7776-qztzm kitchenowl psycopg2.errors.UndefinedTable: relation "household" does not exist
kitchenowl-f9b4d7776-qztzm kitchenowl LINE 2: FROM household
kitchenowl-f9b4d7776-qztzm kitchenowl              ^
kitchenowl-f9b4d7776-qztzm kitchenowl
kitchenowl-f9b4d7776-qztzm kitchenowl
kitchenowl-f9b4d7776-qztzm kitchenowl The above exception was the direct cause of the following exception:
kitchenowl-f9b4d7776-qztzm kitchenowl
kitchenowl-f9b4d7776-qztzm kitchenowl Traceback (most recent call last):
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/usr/src/kitchenowl/upgrade_default_items.py", line 10, in <module>
kitchenowl-f9b4d7776-qztzm kitchenowl     for household in tqdm(Household.query.filter(Household.language != None).all(), desc="Upgrading households..."):
kitchenowl-f9b4d7776-qztzm kitchenowl                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/orm/query.py", line 2688, in all
kitchenowl-f9b4d7776-qztzm kitchenowl     return self._iter().all()  # type: ignore
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/orm/query.py", line 2842, in _iter
kitchenowl-f9b4d7776-qztzm kitchenowl     result: Union[ScalarResult[_T], Result[_T]] = self.session.execute(
kitchenowl-f9b4d7776-qztzm kitchenowl                                                   ^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2262, in execute
kitchenowl-f9b4d7776-qztzm kitchenowl     return self._execute_internal(
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/orm/session.py", line 2144, in _execute_internal
kitchenowl-f9b4d7776-qztzm kitchenowl     result: Result[Any] = compile_state_cls.orm_execute_statement(
kitchenowl-f9b4d7776-qztzm kitchenowl                           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/orm/context.py", line 293, in orm_execute_statement
kitchenowl-f9b4d7776-qztzm kitchenowl     result = conn.execute(
kitchenowl-f9b4d7776-qztzm kitchenowl              ^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1412, in execute
kitchenowl-f9b4d7776-qztzm kitchenowl     return meth(
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/sql/elements.py", line 515, in _execute_on_connection
kitchenowl-f9b4d7776-qztzm kitchenowl     return connection._execute_clauseelement(
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1635, in _execute_clauseelement
kitchenowl-f9b4d7776-qztzm kitchenowl     ret = self._execute_context(
kitchenowl-f9b4d7776-qztzm kitchenowl           ^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1844, in _execute_context
kitchenowl-f9b4d7776-qztzm kitchenowl     return self._exec_single_context(
kitchenowl-f9b4d7776-qztzm kitchenowl            ^^^^^^^^^^^^^^^^^^^^^^^^^^
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1984, in _exec_single_context
kitchenowl-f9b4d7776-qztzm kitchenowl     self._handle_dbapi_exception(
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 2339, in _handle_dbapi_exception
kitchenowl-f9b4d7776-qztzm kitchenowl     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/base.py", line 1965, in _exec_single_context
kitchenowl-f9b4d7776-qztzm kitchenowl     self.dialect.do_execute(
kitchenowl-f9b4d7776-qztzm kitchenowl   File "/opt/venv/lib/python3.11/site-packages/sqlalchemy/engine/default.py", line 921, in do_execute
kitchenowl-f9b4d7776-qztzm kitchenowl     cursor.execute(statement, parameters)
kitchenowl-f9b4d7776-qztzm kitchenowl sqlalchemy.exc.ProgrammingError: (psycopg2.errors.UndefinedTable) relation "household" does not exist
kitchenowl-f9b4d7776-qztzm kitchenowl LINE 2: FROM household
kitchenowl-f9b4d7776-qztzm kitchenowl              ^
kitchenowl-f9b4d7776-qztzm kitchenowl

Share your configuration

using the bitnami postgres helm chart.

deploying this in kubernetes by setting the DB_DRIVER to postgresql

using the default "postgres" admin user so it shouldn't be a permission issue.
from-nibly commented 11 months ago

Looks like downgrading to v76 ended up working for me. Going to try to migrate up to v77 and see if the migration succeeds.

from-nibly commented 11 months ago

yep, as long as I run 76 first, 77 works fine

TomBursch commented 11 months ago

Thanks for the info, I'm currently on vacation and will fix this this weekend.