Upgrading to v3 - failed migrations

bmaximuml commented 7 months ago

Hello 👋

Hope you're doing well @MatthewJohn 😁

When upgrading from v2.81.4 to v3.0.4 I get the following error with MIGRATE_DATABASE set to True:

Log snippet (MIGRATE_DATABASE: True)


terrareg-6c56759c95-fgz25 terrareg INFO  [alembic.runtime.migration] Context impl MySQLImpl.
terrareg-6c56759c95-fgz25 terrareg INFO  [alembic.runtime.migration] Will assume non-transactional DDL.
terrareg-6c56759c95-fgz25 terrareg INFO  [alembic.runtime.migration] Running upgrade fb6a94791a14 -> ee14b27baeb1, Add tables for provider sources, providers and tables for supporting data for provider versions
terrareg-6c56759c95-fgz25 terrareg Traceback (most recent call last):
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
terrareg-6c56759c95-fgz25 terrareg     self.dialect.do_execute(
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
terrareg-6c56759c95-fgz25 terrareg     cursor.execute(statement, parameters)
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/cursor.py", line 615, in execute
terrareg-6c56759c95-fgz25 terrareg     self._handle_result(self._connection.cmd_query(stmt))
terrareg-6c56759c95-fgz25 terrareg                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 1046, in cmd_query
terrareg-6c56759c95-fgz25 terrareg     result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
terrareg-6c56759c95-fgz25 terrareg              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 824, in _handle_result
terrareg-6c56759c95-fgz25 terrareg     raise get_exception(packet)
terrareg-6c56759c95-fgz25 terrareg mysql.connector.errors.ProgrammingError: 1050 (42S01): Table 'provider_analytics' already exists
terrareg-6c56759c95-fgz25 terrareg
terrareg-6c56759c95-fgz25 terrareg The above exception was the direct cause of the following exception:
terrareg-6c56759c95-fgz25 terrareg
terrareg-6c56759c95-fgz25 terrareg Traceback (most recent call last):
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/bin/alembic", line 8, in 
terrareg-6c56759c95-fgz25 terrareg     sys.exit(main())
terrareg-6c56759c95-fgz25 terrareg              ^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/config.py", line 591, in main
terrareg-6c56759c95-fgz25 terrareg     CommandLine(prog=prog).main(argv=argv)
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/config.py", line 585, in main
terrareg-6c56759c95-fgz25 terrareg     self.run_cmd(cfg, options)
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/config.py", line 562, in run_cmd
terrareg-6c56759c95-fgz25 terrareg     fn(
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/command.py", line 378, in upgrade
terrareg-6c56759c95-fgz25 terrareg     script.run_env()
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/script/base.py", line 576, in run_env
terrareg-6c56759c95-fgz25 terrareg     util.load_python_file(self.dir, "env.py")
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 94, in load_python_file
terrareg-6c56759c95-fgz25 terrareg     module = load_module_py(module_id, path)
terrareg-6c56759c95-fgz25 terrareg              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/util/pyfiles.py", line 110, in load_module_py
terrareg-6c56759c95-fgz25 terrareg     spec.loader.exec_module(module)  # type: ignore
terrareg-6c56759c95-fgz25 terrareg     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "", line 994, in exec_module
terrareg-6c56759c95-fgz25 terrareg   File "", line 488, in _call_with_frames_removed
terrareg-6c56759c95-fgz25 terrareg   File "/app/terrareg/alembic/env.py", line 82, in 
terrareg-6c56759c95-fgz25 terrareg     run_migrations_online()
terrareg-6c56759c95-fgz25 terrareg   File "/app/terrareg/alembic/env.py", line 76, in run_migrations_online
terrareg-6c56759c95-fgz25 terrareg     context.run_migrations()
terrareg-6c56759c95-fgz25 terrareg   File "", line 8, in run_migrations
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/runtime/environment.py", line 868, in run_migrations
terrareg-6c56759c95-fgz25 terrareg     self.get_context().run_migrations(**kw)
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/runtime/migration.py", line 622, in run_migrations
terrareg-6c56759c95-fgz25 terrareg     step.migration_fn(**kw)
terrareg-6c56759c95-fgz25 terrareg   File "/app/terrareg/alembic/versions/ee14b27baeb1_add_tables_for_provider_sources_.py", line 21, in upgrade
terrareg-6c56759c95-fgz25 terrareg     op.create_table('provider_analytics',
terrareg-6c56759c95-fgz25 terrareg   File "", line 8, in create_table
terrareg-6c56759c95-fgz25 terrareg   File "", line 3, in create_table
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/operations/ops.py", line 1255, in create_table
terrareg-6c56759c95-fgz25 terrareg     return operations.invoke(op)
terrareg-6c56759c95-fgz25 terrareg            ^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/operations/base.py", line 401, in invoke
terrareg-6c56759c95-fgz25 terrareg     return fn(self, operation)
terrareg-6c56759c95-fgz25 terrareg            ^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/operations/toimpl.py", line 114, in create_table
terrareg-6c56759c95-fgz25 terrareg     operations.impl.create_table(table)
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/ddl/impl.py", line 354, in create_table
terrareg-6c56759c95-fgz25 terrareg     self._exec(schema.CreateTable(table))
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/alembic/ddl/impl.py", line 193, in _exec
terrareg-6c56759c95-fgz25 terrareg     return conn.execute(  # type: ignore[call-overload]
terrareg-6c56759c95-fgz25 terrareg            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
terrareg-6c56759c95-fgz25 terrareg     return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
terrareg-6c56759c95-fgz25 terrareg            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/ddl.py", line 80, in _execute_on_connection
terrareg-6c56759c95-fgz25 terrareg     return connection._execute_ddl(
terrareg-6c56759c95-fgz25 terrareg            ^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1472, in _execute_ddl
terrareg-6c56759c95-fgz25 terrareg     ret = self._execute_context(
terrareg-6c56759c95-fgz25 terrareg           ^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
terrareg-6c56759c95-fgz25 terrareg     self._handle_dbapi_exception(
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
terrareg-6c56759c95-fgz25 terrareg     util.raise_(
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
terrareg-6c56759c95-fgz25 terrareg     raise exception
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
terrareg-6c56759c95-fgz25 terrareg     self.dialect.do_execute(
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
terrareg-6c56759c95-fgz25 terrareg     cursor.execute(statement, parameters)
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/cursor.py", line 615, in execute
terrareg-6c56759c95-fgz25 terrareg     self._handle_result(self._connection.cmd_query(stmt))
terrareg-6c56759c95-fgz25 terrareg                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 1046, in cmd_query
terrareg-6c56759c95-fgz25 terrareg     result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
terrareg-6c56759c95-fgz25 terrareg              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-6c56759c95-fgz25 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 824, in _handle_result
terrareg-6c56759c95-fgz25 terrareg     raise get_exception(packet)
terrareg-6c56759c95-fgz25 terrareg sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1050 (42S01): Table 'provider_analytics' already exists
terrareg-6c56759c95-fgz25 terrareg [SQL:
terrareg-6c56759c95-fgz25 terrareg CREATE TABLE provider_analytics (
terrareg-6c56759c95-fgz25 terrareg  id INTEGER NOT NULL AUTO_INCREMENT,
terrareg-6c56759c95-fgz25 terrareg  provider_version_id INTEGER NOT NULL,
terrareg-6c56759c95-fgz25 terrareg  timestamp DATETIME,
terrareg-6c56759c95-fgz25 terrareg  terraform_version VARCHAR(128),
terrareg-6c56759c95-fgz25 terrareg  namespace_name VARCHAR(128),
terrareg-6c56759c95-fgz25 terrareg  provider_name VARCHAR(128),
terrareg-6c56759c95-fgz25 terrareg  PRIMARY KEY (id)
terrareg-6c56759c95-fgz25 terrareg )
terrareg-6c56759c95-fgz25 terrareg
terrareg-6c56759c95-fgz25 terrareg ]
terrareg-6c56759c95-fgz25 terrareg (Background on this error at: https://sqlalche.me/e/14/f405)

and the following with it set to False:

Log snippet (MIGRATE_DATABASE: False)


terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,284 INFO sqlalchemy.engine.Engine SELECT DATABASE()
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,284 INFO sqlalchemy.engine.Engine [raw sql] {}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,289 INFO sqlalchemy.engine.Engine SELECT @@sql_mode
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,289 INFO sqlalchemy.engine.Engine [raw sql] {}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,293 INFO sqlalchemy.engine.Engine SELECT @@lower_case_table_names
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,293 INFO sqlalchemy.engine.Engine [raw sql] {}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,301 INFO sqlalchemy.engine.Engine SELECT git_provider.id, git_provider.name, git_provider.base_url_template, git_provider.clone_url_template, git_provider.browse_url_template
terrareg-ddcd67f55-n2c87 terrareg FROM git_provider
terrareg-ddcd67f55-n2c87 terrareg WHERE git_provider.name = %(name_1)s
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,301 INFO sqlalchemy.engine.Engine [generated in 0.00237s] {'name_1': 'GitHub'}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,310 INFO sqlalchemy.engine.Engine SELECT git_provider.id, git_provider.name, git_provider.base_url_template, git_provider.clone_url_template, git_provider.browse_url_template
terrareg-ddcd67f55-n2c87 terrareg FROM git_provider
terrareg-ddcd67f55-n2c87 terrareg WHERE git_provider.id = %(id_1)s
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,310 INFO sqlalchemy.engine.Engine [generated in 0.00173s] {'id_1': 1}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,317 INFO sqlalchemy.engine.Engine UPDATE git_provider SET base_url_template=%(base_url_template)s, clone_url_template=%(clone_url_template)s, browse_url_template=%(browse_url_template)s WHERE git_provider.id = %(id_1)s
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,317 INFO sqlalchemy.engine.Engine [generated in 0.00174s] {'base_url_template': 'https://github.com/{namespace}/{module}', 'clone_url_template': 'ssh://git@github.com/{namespace}/{module}.git', 'browse_url_template': 'https://github.com/{namespace}/{module}/tree/{tag}/{path}', 'id_1': 1}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,319 INFO sqlalchemy.engine.Engine COMMIT
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,325 INFO sqlalchemy.engine.Engine BEGIN (implicit)
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,330 INFO sqlalchemy.engine.Engine SELECT provider_category.id, provider_category.name, provider_category.slug, provider_category.user_selectable
terrareg-ddcd67f55-n2c87 terrareg FROM provider_category
terrareg-ddcd67f55-n2c87 terrareg WHERE provider_category.id = %(id_1)s
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,330 INFO sqlalchemy.engine.Engine [generated in 0.00273s] {'id_1': 1}
terrareg-ddcd67f55-n2c87 terrareg 2024-02-12 17:05:56,359 INFO sqlalchemy.engine.Engine ROLLBACK
terrareg-ddcd67f55-n2c87 terrareg Traceback (most recent call last):
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
terrareg-ddcd67f55-n2c87 terrareg     self.dialect.do_execute(
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
terrareg-ddcd67f55-n2c87 terrareg     cursor.execute(statement, parameters)
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/cursor.py", line 615, in execute
terrareg-ddcd67f55-n2c87 terrareg     self._handle_result(self._connection.cmd_query(stmt))
terrareg-ddcd67f55-n2c87 terrareg                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 1046, in cmd_query
terrareg-ddcd67f55-n2c87 terrareg     result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
terrareg-ddcd67f55-n2c87 terrareg              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 824, in _handle_result
terrareg-ddcd67f55-n2c87 terrareg     raise get_exception(packet)
terrareg-ddcd67f55-n2c87 terrareg mysql.connector.errors.ProgrammingError: 1146 (42S02): Table 'terrareg-database.provider_category' doesn't exist
terrareg-ddcd67f55-n2c87 terrareg
terrareg-ddcd67f55-n2c87 terrareg The above exception was the direct cause of the following exception:
terrareg-ddcd67f55-n2c87 terrareg
terrareg-ddcd67f55-n2c87 terrareg Traceback (most recent call last):
terrareg-ddcd67f55-n2c87 terrareg   File "/app/./terrareg.py", line 20, in 
terrareg-ddcd67f55-n2c87 terrareg     s = Server(ssl_public_key=args.ssl_pub_key, ssl_private_key=args.ssl_priv_key)
terrareg-ddcd67f55-n2c87 terrareg         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/app/terrareg/server/__init__.py", line 139, in __init__
terrareg-ddcd67f55-n2c87 terrareg     terrareg.provider_category_model.ProviderCategoryFactory.get().initialise_from_config()
terrareg-ddcd67f55-n2c87 terrareg   File "/app/terrareg/provider_category_model.py", line 192, in initialise_from_config
terrareg-ddcd67f55-n2c87 terrareg     existing_provider_category = self.get_provider_category_by_pk(pk=pk)
terrareg-ddcd67f55-n2c87 terrareg                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/app/terrareg/provider_category_model.py", line 126, in get_provider_category_by_pk
terrareg-ddcd67f55-n2c87 terrareg     if instance.exists:
terrareg-ddcd67f55-n2c87 terrareg        ^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/app/terrareg/provider_category_model.py", line 43, in exists
terrareg-ddcd67f55-n2c87 terrareg     return bool(self._get_db_row())
terrareg-ddcd67f55-n2c87 terrareg                 ^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/app/terrareg/provider_category_model.py", line 70, in _get_db_row
terrareg-ddcd67f55-n2c87 terrareg     res = conn.execute(select)
terrareg-ddcd67f55-n2c87 terrareg           ^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1380, in execute
terrareg-ddcd67f55-n2c87 terrareg     return meth(self, multiparams, params, _EMPTY_EXECUTION_OPTS)
terrareg-ddcd67f55-n2c87 terrareg            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/sql/elements.py", line 334, in _execute_on_connection
terrareg-ddcd67f55-n2c87 terrareg     return connection._execute_clauseelement(
terrareg-ddcd67f55-n2c87 terrareg            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement
terrareg-ddcd67f55-n2c87 terrareg     ret = self._execute_context(
terrareg-ddcd67f55-n2c87 terrareg           ^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context
terrareg-ddcd67f55-n2c87 terrareg     self._handle_dbapi_exception(
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2124, in _handle_dbapi_exception
terrareg-ddcd67f55-n2c87 terrareg     util.raise_(
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/util/compat.py", line 211, in raise_
terrareg-ddcd67f55-n2c87 terrareg     raise exception
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context
terrareg-ddcd67f55-n2c87 terrareg     self.dialect.do_execute(
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 736, in do_execute
terrareg-ddcd67f55-n2c87 terrareg     cursor.execute(statement, parameters)
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/cursor.py", line 615, in execute
terrareg-ddcd67f55-n2c87 terrareg     self._handle_result(self._connection.cmd_query(stmt))
terrareg-ddcd67f55-n2c87 terrareg                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 1046, in cmd_query
terrareg-ddcd67f55-n2c87 terrareg     result = self._handle_result(self._send_cmd(ServerCmd.QUERY, query))
terrareg-ddcd67f55-n2c87 terrareg              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
terrareg-ddcd67f55-n2c87 terrareg   File "/usr/local/lib/python3.12/site-packages/mysql/connector/connection.py", line 824, in _handle_result
terrareg-ddcd67f55-n2c87 terrareg     raise get_exception(packet)
terrareg-ddcd67f55-n2c87 terrareg sqlalchemy.exc.ProgrammingError: (mysql.connector.errors.ProgrammingError) 1146 (42S02): Table 'terrareg-database.provider_category' doesn't exist
terrareg-ddcd67f55-n2c87 terrareg [SQL: SELECT provider_category.id, provider_category.name, provider_category.slug, provider_category.user_selectable
terrareg-ddcd67f55-n2c87 terrareg FROM provider_category
terrareg-ddcd67f55-n2c87 terrareg WHERE provider_category.id = %(id_1)s]
terrareg-ddcd67f55-n2c87 terrareg [parameters: {'id_1': 1}]
terrareg-ddcd67f55-n2c87 terrareg (Background on this error at: https://sqlalche.me/e/14/f405)

Any way I can upgrade without losing data?

MatthewJohn commented 7 months ago

Hey @bmaximuml,

I'm good thanks - hope you're doing well (aside from this! :eyes: )..

I took a quick look, there's only one migration that tries to add the table provider_analytics, which is odd - so two things I'd like to ask:

Did you have an instance that attempted the migration beforehand?
Do you run multiple Terrareg containers normally?

Matt

bmaximuml commented 7 months ago

Did you have an instance that attempted the migration beforehand?

Probably, have attempted to run migrations a few times now

Do you run multiple Terrareg containers normally?

Nope, just the one

MatthewJohn commented 7 months ago

So, if there was another issue with the initial migration, will need to try to re-create that. This migration is "quite a nice one", in that it creates a bunch of tables, with the caveat of the last item that (for MySQL), it updates the AUDIT enum.

If it did fail for another reason, it very possible that a second run would yield the error that you received (because it's the first thing it tries to do).

Before attempting anything, it would be good to determine where the original migration go to... can you see which of these tables exist?

provider_analytics
provider_category
provider_source
gpg_key
repository
provider
provider_version
provider_version_binary
provider_version_documentation

(without knowing anything about your MySQL knowledge, which I'm not going to try and guess :)). If you login to the database, run: use terrareg; (where terrareg is the database name and describe table_name; where "table_name" is one of the tables from the above list).

If those tables do all exist, could you:

check to see if the provider_source_auth column exists in the session table.
Check the definition of the action column of the audit_history table (and send it across, if you could)

:)

Out of interest, do you have any healthchecks on the container, that would kill it if it wasn't healthy in a particular time?

bmaximuml commented 7 months ago

Here're the tables in the db:

+----------------------------------+
| Tables_in_terrareg-database      |
+----------------------------------+
| alembic_version                  |
| analytics                        |
| audit_history                    |
| example_file                     |
| git_provider                     |
| module_details                   |
| module_provider                  |
| module_provider_redirect         |
| module_version                   |
| module_version_file              |
| namespace                        |
| namespace_redirect               |
| provider_analytics               |
| session                          |
| submodule                        |
| terraform_idp_access_token       |
| terraform_idp_authorization_code |
| terraform_idp_subject_identifier |
| user_group                       |
| user_group_namespace_permission  |
+----------------------------------+

provider_analytics YES
provider_category NO
provider_source NO
gpg_key NO
repository NO
provider NO
provider_version NO
provider_version_binary NO
provider_version_documentation NO

Healthchecks: I don't think so

bmaximuml commented 7 months ago

Also - how does PROVIDER_SOURCES differ from GIT_PROVIDER_CONFIG?

bmaximuml commented 7 months ago

Managed to fix the migrations issue - I renamed the provider_analytics and provider_category tables (to provider_analytics_old and provider_category_old) then manually ran alembic upgrade head. Fortunately, I don't have much data in those tables so I'm not too sad about losing it, although I guess if I did then I could try migrating the data now the app is working.

Thanks for looking into this!

I'm gonna close this issue and open a fresh one for my question above

MatthewJohn commented 7 months ago

That's very interesting - those tables are brand new in the migration - not sure how they ended up with data in them!

MatthewJohn commented 6 months ago

Created gitlab issue: https://gitlab.dockstudios.co.uk/pub/terrareg/-/issues/508 gitlab-issue-id:508

MatthewJohn / terrareg

Upgrading to v3 - failed migrations #34