Closed gcarrarom closed 2 months ago
BeryJu
commented
4 months ago Could you try to upgrade to 2024.4.1 and then to 2024.4.2? We test version upgrades as part of our CI pipeline, however we only test upgrades from the immediate previous version
gcarrarom
commented
4 months ago Thanks for the heads-up on that! I've upgraded without issues to 2024.4.1, but from 2024.4.1 to 2024.4.2 it fails with the same issue:
"pid": 32, "schema_name": "public", "timestamp": "2024-05-13T18:40:08.221343"}
{"app_name": "authentik.events", "domain_url": null, "event": "Imported related module", "level": "info", "logger": "authentik.blueprints.apps", "module": "authentik.events.signals", "pid": 32, "schema_name": "public", "timestamp": "2024-05-13T18:40:08.221450"}
=== Starting migration
Operations to perform:
Apply all migrations: auth, authentik_blueprints, authentik_brands, authentik_core, authentik_crypto, authentik_enterprise, authentik_events, authentik_flows, authentik_outposts, authentik_policies, authentik_policies_dummy, authentik_policies_event_matcher, authentik_policies_expiry, authentik_policies_expression, authentik_policies_password, authentik_policies_reputation, authentik_providers_ldap, authentik_providers_oauth2, authentik_providers_proxy, authentik_providers_rac, authentik_providers_radius, authentik_providers_saml, authentik_providers_scim, authentik_rbac, authentik_sources_ldap, authentik_sources_oauth, authentik_sources_plex, authentik_sources_saml, authentik_sources_scim, authentik_stages_authenticator_duo, authentik_stages_authenticator_sms, authentik_stages_authenticator_static, authentik_stages_authenticator_totp, authentik_stages_authenticator_validate, authentik_stages_authenticator_webauthn, authentik_stages_captcha, authentik_stages_consent, authentik_stages_deny, authentik_stages_dummy, authentik_stages_email, authentik_stages_identification, authentik_stages_invitation, authentik_stages_password, authentik_stages_prompt, authentik_stages_source, authentik_stages_user_delete, authentik_stages_user_login, authentik_stages_user_logout, authentik_stages_user_write, authentik_tenants, contenttypes, guardian, sessions
Running migrations:
Applying authentik_providers_scim.0007_scimgroup_scim_id_scimuser_scim_id_and_more...
Fixing primary key for SCIM users, this might take a couple of minutes...
Fixing primary key for SCIM groups, this might take a couple of minutes...
{"domain_url": null, "event": "releasing database lock", "level": "info", "logger": "lifecycle.migrate", "pid": 32, "schema_name": "public", "timestamp": "2024-05-13T18:40:10.671378"}
Failed to read config file: ./lifecycle/gunicorn.conf.py
Traceback (most recent call last):
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 103, in _execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django_prometheus/db/common.py", line 69, in execute
return super().execute(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/psycopg/cursor.py", line 732, in execute
raise ex.with_traceback(None)
psycopg.errors.UndefinedObject: constraint "authentik_providers_scim_id_group_id_provider_id_9d50d292_uniq" of relation "authentik_providers_scim_scimgroup" does not exist
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/ak-root/venv/lib/python3.12/site-packages/gunicorn/app/base.py", line 111, in get_config_from_filename
spec.loader.exec_module(mod)
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/lifecycle/gunicorn.conf.py", line 147, in <module>
run_migrations()
File "/lifecycle/migrate.py", line 113, in run_migrations
execute_from_command_line(["", "migrate_schemas"])
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
utility.execute()
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 413, in run_from_argv
self.execute(*args, **cmd_options)
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django_tenants/management/commands/migrate_schemas.py", line 63, in handle
executor.run_migrations(tenants=[self.PUBLIC_SCHEMA_NAME])
File "/ak-root/venv/lib/python3.12/site-packages/django_tenants/migration_executors/standard.py", line 11, in run_migrations
run_migrations(self.args, self.options, self.codename, self.PUBLIC_SCHEMA_NAME)
File "/ak-root/venv/lib/python3.12/site-packages/django_tenants/migration_executors/base.py", line 59, in run_migrations
migrate_command_class(stdout=stdout, stderr=stderr).execute(*args, **options)
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 459, in execute
output = self.handle(*args, **options)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 107, in wrapper
res = handle_func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 356, in handle
post_migrate_state = executor.migrate(
^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 135, in migrate
state = self._migrate_all_forwards(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
state = self.apply_migration(
^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
state = migration.apply(state, schema_editor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/migration.py", line 132, in apply
operation.database_forwards(
File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/operations/models.py", line 659, in database_forwards
alter_together(
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/base/schema.py", line 594, in alter_unique_together
self._delete_composed_index(
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/base/schema.py", line 658, in _delete_composed_index
self.execute(self._delete_constraint_sql(sql, model, constraint_names[0]))
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/postgresql/schema.py", line 48, in execute
return super().execute(sql, None)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/base/schema.py", line 201, in execute
cursor.execute(sql, params)
File "/ak-root/venv/lib/python3.12/site-packages/sentry_sdk/integrations/django/__init__.py", line 644, in execute
result = real_execute(self, sql, params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 79, in execute
return self._execute_with_wrappers(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
return executor(sql, params, many, context)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 100, in _execute
with self.db.wrap_database_errors:
File "/ak-root/venv/lib/python3.12/site-packages/django/db/utils.py", line 91, in __exit__
raise dj_exc_value.with_traceback(traceback) from exc_value
File "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py", line 103, in _execute
return self.cursor.execute(sql)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/django_prometheus/db/common.py", line 69, in execute
return super().execute(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/ak-root/venv/lib/python3.12/site-packages/psycopg/cursor.py", line 732, in execute
raise ex.with_traceback(None)
django.db.utils.ProgrammingError: constraint "authentik_providers_scim_id_group_id_provider_id_9d50d292_uniq" of relation "authentik_providers_scim_scimgroup" does not existAnyone? Same issue trying to go to 2024.6.0...
PracticalRyan
commented
3 months ago I'm using a regular docker compose file upgrading from 2024.4.3 to 2024.6.0 and I'm getting a similar error related to Failed to read config file: ./lifecycle/gunicorn.conf.py I have already followed the PostgreSQL 16 update steps for 2024.6.0, but I'm still having issues. Reverting back works totally fine though but only breaks when updating.
Not sure if it's related though.
gcarrarom
commented
2 months ago Yeah, doesn't seem to be the same issue. I'm running PSQL 16 as well, version 16.3. Could it be something in the version of PostGreSQL?
PracticalRyan
commented
2 months ago I have no idea what that issue was, but loosely following this guide. I was able to fix the issue. I first created a dump of the database, downed and deleted all containers, create temporary bind mounts to empty directories. (At first I just deleted the DB volume but it still errored, so it had to be more than just the DB, which is why I deleted everything from the containers.)
After doing all this, docker compose up and you'll get the initial Authentik instance with no data whatsoever. Now that it runs on the latest version, I moved back all my bind mounts to their original places, restarted, and everything still worked! But since all my data was gone, I just applied the DB dump to the empty database and now it runs normally.
gcarrarom
commented
2 months ago Alright, thanks for letting me know. I've fixed this issue by manually doing the migrations from: https://github.com/goauthentik/authentik/tree/main/authentik/providers/scim/migrations
I don't know how that migration was broken, but manually migrating tables and altering them in a DB is not ideal. Once I've done the steps on those python migration scripts, and I've added the migration names in the django_migrations table, I now have version 2024.6.1 up and running.
I do not recommend someone else doing this without understanding how those scripts happen and how they are supposed to run. But it did work.
I've tried running a new version of authentik in a fresh DB and migrating the data over, but the number of constraints and the new IDs were way too much for me to migrate over. I'll close for now, but I'm not happy that I had to do this.
While upgrading to 2024.4.2 the Server and worker pods fail with the following issue:
Rollback to the 2024.2.2 version works just fine.
To Reproduce
Expected behavior I'd expect the upgrade of the app, it'll take care of missing relationships in the Database and ensure that it would work out of the box.
Logs
Version and Deployment: