search

goauthentik / authentik

The authentication glue you need.
https://goauthentik.io
Other
13.27k stars 884 forks source link

Upgrade to 2024.8.0 fails in database migration: AttributeError: 'NoneType' object has no attribute 'users' #11171

Closed jgraichen closed 1 month ago

jgraichen commented 1 month ago

Describe the bug

Upgraded from 2024.6.4 to 2024.8.0, database migrations fail inside container:

Sep 03 13:40:19 authentik 4345478311a4[848]: === Starting migration
Sep 03 13:40:19 authentik 4345478311a4[848]: Operations to perform:
Sep 03 13:40:19 authentik 4345478311a4[848]:   Apply all migrations: auth, authentik_blueprints, authentik_brands, authentik_core, authentik_crypto, authentik_enterprise, authentik_events, authentik_flows, authentik_outposts, authentik_policies, authentik_policies_dummy, authentik_policies_event_matcher, authentik_policies_expiry, authentik_policies_expression, authentik_policies_geoip, authentik_policies_password, authentik_policies_reputation, authentik_providers_google_workspace, authentik_providers_ldap, authentik_providers_microsoft_entra, authentik_providers_oauth2, authentik_providers_proxy, authentik_providers_rac, authentik_providers_radius, authentik_providers_saml, authentik_providers_scim, authentik_rbac, authentik_sources_ldap, authentik_sources_oauth, authentik_sources_plex, authentik_sources_saml, authentik_sources_scim, authentik_stages_authenticator_duo, authentik_stages_authenticator_sms, authentik_stages_authenticator_static, authentik_stages_authenticator_totp, authentik_stages_authenticator_validate, authentik_stages_authenticator_webauthn, authentik_stages_captcha, authentik_stages_consent, authentik_stages_deny, authentik_stages_dummy, authentik_stages_email, authentik_stages_identification, authentik_stages_invitation, authentik_stages_password, authentik_stages_prompt, authentik_stages_source, authentik_stages_user_delete, authentik_stages_user_login, authentik_stages_user_logout, authentik_stages_user_write, authentik_tenants, contenttypes, guardian, sessions
Sep 03 13:40:19 authentik 4345478311a4[848]: Running migrations:
Sep 03 13:40:19 authentik 4345478311a4[848]:   Applying authentik_providers_ldap.0004_alter_ldapprovider_options_and_more...
Sep 03 13:40:19 authentik 4345478311a4[848]: Adding permission 'authentik Providers.LDAP | LDAP Provider | Search full LDAP directory'
Sep 03 13:40:19 authentik 4345478311a4[848]: {"domain_url": null, "event": "releasing database lock", "level": "info", "logger": "lifecycle.migrate", "pid": 7, "schema_name": "public", "timestamp": "2024-09-03T13:40:19.750970"}
Sep 03 13:40:19 authentik 4345478311a4[848]: Traceback (most recent call last):
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "<frozen runpy>", line 198, in _run_module_as_main
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "<frozen runpy>", line 88, in _run_code
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/manage.py", line 43, in <module>
Sep 03 13:40:19 authentik 4345478311a4[848]:     run_migrations()
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/lifecycle/migrate.py", line 112, in run_migrations
Sep 03 13:40:19 authentik 4345478311a4[848]:     execute_from_command_line(["", "migrate_schemas"])
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 442, in execute_from_command_line
Sep 03 13:40:19 authentik 4345478311a4[848]:     utility.execute()
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/__init__.py", line 436, in execute
Sep 03 13:40:19 authentik 4345478311a4[848]:     self.fetch_command(subcommand).run_from_argv(self.argv)
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 413, in run_from_argv
Sep 03 13:40:19 authentik 4345478311a4[848]:     self.execute(*args, **cmd_options)
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 459, in execute
Sep 03 13:40:19 authentik 4345478311a4[848]:     output = self.handle(*args, **options)
Sep 03 13:40:19 authentik 4345478311a4[848]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django_tenants/management/commands/migrate_schemas.py", line 63, in handle
Sep 03 13:40:19 authentik 4345478311a4[848]:     executor.run_migrations(tenants=[self.PUBLIC_SCHEMA_NAME])
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django_tenants/migration_executors/standard.py", line 11, in run_migrations
Sep 03 13:40:19 authentik 4345478311a4[848]:     run_migrations(self.args, self.options, self.codename, self.PUBLIC_SCHEMA_NAME)
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django_tenants/migration_executors/base.py", line 59, in run_migrations
Sep 03 13:40:19 authentik 4345478311a4[848]:     migrate_command_class(stdout=stdout, stderr=stderr).execute(*args, **options)
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 459, in execute
Sep 03 13:40:19 authentik 4345478311a4[848]:     output = self.handle(*args, **options)
Sep 03 13:40:19 authentik 4345478311a4[848]:              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/base.py", line 107, in wrapper
Sep 03 13:40:19 authentik 4345478311a4[848]:     res = handle_func(*args, **kwargs)
Sep 03 13:40:19 authentik 4345478311a4[848]:           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/core/management/commands/migrate.py", line 356, in handle
Sep 03 13:40:19 authentik 4345478311a4[848]:     post_migrate_state = executor.migrate(
Sep 03 13:40:19 authentik 4345478311a4[848]:                          ^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 135, in migrate
Sep 03 13:40:19 authentik 4345478311a4[848]:     state = self._migrate_all_forwards(
Sep 03 13:40:19 authentik 4345478311a4[848]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards
Sep 03 13:40:19 authentik 4345478311a4[848]:     state = self.apply_migration(
Sep 03 13:40:19 authentik 4345478311a4[848]:             ^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/executor.py", line 252, in apply_migration
Sep 03 13:40:19 authentik 4345478311a4[848]:     state = migration.apply(state, schema_editor)
Sep 03 13:40:19 authentik 4345478311a4[848]:             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/migration.py", line 132, in apply
Sep 03 13:40:19 authentik 4345478311a4[848]:     operation.database_forwards(
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/ak-root/venv/lib/python3.12/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards
Sep 03 13:40:19 authentik 4345478311a4[848]:     self.code(from_state.apps, schema_editor)
Sep 03 13:40:19 authentik 4345478311a4[848]:   File "/authentik/providers/ldap/migrations/0004_alter_ldapprovider_options_and_more.py", line 26, in migrate_search_group
Sep 03 13:40:19 authentik 4345478311a4[848]:     provider.search_group.users.using(db_alias).all().values_list("pk", flat=True)
Sep 03 13:40:19 authentik 4345478311a4[848]:     ^^^^^^^^^^^^^^^^^^^^^^^^^^^
Sep 03 13:40:19 authentik 4345478311a4[848]: AttributeError: 'NoneType' object has no attribute 'users'

To Reproduce

Steps to reproduce the behavior:

  1. Have 2024.6.4 running
  2. Deploy 2024.8.0
  3. See error in docker logs

Expected behavior

Database migrations passing and application starting.

Logs

See above.

Version and Deployment (please complete the following information):

Additional context

This authentik does not have any LDAP source configured.

BeryJu commented 1 month ago

Fixed by https://github.com/goauthentik/authentik/pull/11170

jgraichen commented 1 month ago

@BeryJu Thanks for fixing! Will there be an updated 2024.8 image available soon?

I'd like to avoid restoring everything from a backup, but a simple rollback of the image only does not work:

{
    "event": "Internal Server Error: /-/health/live/",
    "exception": [
        {
            "exc_type": "ProgrammingError",
            "exc_value": 'column authentik_enterprise_licenseusage.user_count does not exist\nLINE 1: ..."authentik_enterprise_licenseusage"."usage_uuid", "authentik...\n                                                             ^',
            "frames": [
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/core/handlers/exception.py",
                    "line": "",
                    "lineno": 55,
                    "locals": {
                        "exc": '"ProgrammingError(\'column authentik_enterprise_licenseusage.user_count does not e"+146',
                        "get_response": "'<authentik.enterprise.audit.middleware.EnterpriseAuditMiddleware object at 0x730'+10",
                        "request": "<ASGIRequest: HEAD '/-/health/live/'>",
                    },
                    "name": "inner",
                },
                {
                    "filename": "/authentik/events/middleware.py",
                    "line": "",
                    "lineno": 154,
                    "locals": {
                        "request": "<ASGIRequest: HEAD '/-/health/live/'>",
                        "self": "'<authentik.enterprise.audit.middleware.EnterpriseAuditMiddleware object at 0x730'+10",
                    },
                    "name": "__call__",
                },
                {
                    "filename": "/authentik/enterprise/audit/middleware.py",
                    "line": "",
                    "lineno": 29,
                    "locals": {
                        "__class__": "<class 'authentik.enterprise.audit.middleware.EnterpriseAuditMiddleware'>",
                        "request": "<ASGIRequest: HEAD '/-/health/live/'>",
                        "self": "'<authentik.enterprise.audit.middleware.EnterpriseAuditMiddleware object at 0x730'+10",
                    },
                    "name": "connect",
                },
                {
                    "filename": "/authentik/enterprise/audit/middleware.py",
                    "line": "",
                    "lineno": 25,
                    "locals": {
                        "self": "'<authentik.enterprise.audit.middleware.EnterpriseAuditMiddleware object at 0x730'+10"
                    },
                    "name": "enabled",
                },
                {
                    "filename": "/authentik/enterprise/apps.py",
                    "line": "",
                    "lineno": 22,
                    "locals": {
                        "self": "<AuthentikEnterpriseConfig: authentik_enterprise>"
                    },
                    "name": "enabled",
                },
                {
                    "filename": "/authentik/enterprise/apps.py",
                    "line": "",
                    "lineno": 28,
                    "locals": {
                        "LicenseKey": "<class 'authentik.enterprise.license.LicenseKey'>",
                        "self": "<AuthentikEnterpriseConfig: authentik_enterprise>",
                    },
                    "name": "check_enabled",
                },
                {
                    "filename": "/authentik/enterprise/license.py",
                    "line": "",
                    "lineno": 207,
                    "locals": {"summary": "None"},
                    "name": "cached_summary",
                },
                {
                    "filename": "/authentik/enterprise/license.py",
                    "line": "",
                    "lineno": 186,
                    "locals": {
                        "has_license": "False",
                        "self": '"LicenseKey(aud=\'enterprise.goauthentik.io/license/cf9829b0-876d-402a-b94e-8a4196"+88',
                    },
                    "name": "summary",
                },
                {
                    "filename": "/authentik/enterprise/license.py",
                    "line": "",
                    "lineno": 177,
                    "locals": {},
                    "name": "last_valid_date",
                },
                {
                    "filename": "/authentik/core/models.py",
                    "line": "",
                    "lineno": 678,
                    "locals": {
                        "cls": "<class 'authentik.enterprise.models.LicenseUsage'>",
                        "kwargs": "{'within_limits': True}",
                    },
                    "name": "filter_not_expired",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/models/query.py",
                    "line": "",
                    "lineno": 400,
                    "locals": {
                        "self": '"<repr-error \'column authentik_enterprise_licenseusage.user_count does not exist\\\\"+141'
                    },
                    "name": "__iter__",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/models/query.py",
                    "line": "",
                    "lineno": 1928,
                    "locals": {
                        "self": '"<repr-error \'column authentik_enterprise_licenseusage.user_count does not exist\\\\"+141'
                    },
                    "name": "_fetch_all",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/models/query.py",
                    "line": "",
                    "lineno": 91,
                    "locals": {
                        "compiler": "\"<SQLCompiler model=LicenseUsage connection=<DatabaseWrapper vendor='postgresql' \"+33",
                        "db": "default",
                        "queryset": '"<repr-error \'column authentik_enterprise_licenseusage.user_count does not exist\\\\"+141',
                        "self": "<django.db.models.query.ModelIterable object at 0x730e821dcc20>",
                    },
                    "name": "__iter__",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/models/sql/compiler.py",
                    "line": "",
                    "lineno": 1562,
                    "locals": {
                        "chunk_size": "100",
                        "chunked_fetch": "False",
                        "cursor": "<django.db.backends.utils.CursorWrapper object at 0x730e82180350>",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "result_type": "multi",
                        "self": "\"<SQLCompiler model=LicenseUsage connection=<DatabaseWrapper vendor='postgresql' \"+33",
                        "sql": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                    },
                    "name": "execute_sql",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py",
                    "line": "",
                    "lineno": 79,
                    "locals": {
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "self": "<django.db.backends.utils.CursorWrapper object at 0x730e82180350>",
                        "sql": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                    },
                    "name": "execute",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py",
                    "line": "",
                    "lineno": 92,
                    "locals": {
                        "context": "\"{'connection': <DatabaseWrapper vendor='postgresql' alias='default'>, 'cursor': \"+66",
                        "executor": "'<bound method CursorWrapper._execute of <django.db.backends.utils.CursorWrapper '+26",
                        "many": "False",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "self": "<django.db.backends.utils.CursorWrapper object at 0x730e82180350>",
                        "sql": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                    },
                    "name": "_execute_with_wrappers",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py",
                    "line": "",
                    "lineno": 100,
                    "locals": {
                        "ignored_wrapper_args": "\"(False, {'connection': <DatabaseWrapper vendor='postgresql' alias='default'>, 'c\"+75",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "self": "<django.db.backends.utils.CursorWrapper object at 0x730e82180350>",
                        "sql": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                    },
                    "name": "_execute",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/utils.py",
                    "line": "",
                    "lineno": 91,
                    "locals": {
                        "db_exc_type": "<class 'psycopg.ProgrammingError'>",
                        "dj_exc_type": "<class 'django.db.utils.ProgrammingError'>",
                        "dj_exc_value": '"ProgrammingError(\'column authentik_enterprise_licenseusage.user_count does not e"+146',
                        "exc_type": "<class 'psycopg.errors.UndefinedColumn'>",
                        "exc_value": '"UndefinedColumn(\'column authentik_enterprise_licenseusage.user_count does not ex"+145',
                        "self": "<django.db.utils.DatabaseErrorWrapper object at 0x730e821dc050>",
                        "traceback": "<traceback object at 0x730e821cdc00>",
                    },
                    "name": "__exit__",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py",
                    "line": "",
                    "lineno": 105,
                    "locals": {
                        "ignored_wrapper_args": "\"(False, {'connection': <DatabaseWrapper vendor='postgresql' alias='default'>, 'c\"+75",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "self": "<django.db.backends.utils.CursorWrapper object at 0x730e82180350>",
                        "sql": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                    },
                    "name": "_execute",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django_prometheus/db/common.py",
                    "line": "",
                    "lineno": 69,
                    "locals": {
                        "__class__": '"<class \'django_prometheus.db.common.ExportingCursorWrapper.<locals>.CursorWrappe"+3',
                        "alias": "default",
                        "args": '\'(\\\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_l\'+578',
                        "kwargs": "{}",
                        "labels": "{'alias': 'default', 'vendor': 'postgresql'}",
                        "self": "'<django_prometheus.db.common.ExportingCursorWrapper.<locals>.CursorWrapper [clos'+50",
                        "vendor": "postgresql",
                    },
                    "name": "execute",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/psycopg/cursor.py",
                    "line": "",
                    "lineno": 732,
                    "locals": {
                        "binary": "None",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "prepare": "None",
                        "query": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                        "self": "'<django_prometheus.db.common.ExportingCursorWrapper.<locals>.CursorWrapper [clos'+50",
                    },
                    "name": "execute",
                },
            ],
            "is_cause": false,
            "syntax_error": null,
        },
        {
            "exc_type": "UndefinedColumn",
            "exc_value": 'column authentik_enterprise_licenseusage.user_count does not exist\nLINE 1: ..."authentik_enterprise_licenseusage"."usage_uuid", "authentik...\n                                                             ^',
            "frames": [
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django/db/backends/utils.py",
                    "line": "",
                    "lineno": 105,
                    "locals": {
                        "ignored_wrapper_args": "\"(False, {'connection': <DatabaseWrapper vendor='postgresql' alias='default'>, 'c\"+75",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "self": "<django.db.backends.utils.CursorWrapper object at 0x730e82180350>",
                        "sql": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                    },
                    "name": "_execute",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/django_prometheus/db/common.py",
                    "line": "",
                    "lineno": 69,
                    "locals": {
                        "__class__": '"<class \'django_prometheus.db.common.ExportingCursorWrapper.<locals>.CursorWrappe"+3',
                        "alias": "default",
                        "args": '\'(\\\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_l\'+578',
                        "kwargs": "{}",
                        "labels": "{'alias': 'default', 'vendor': 'postgresql'}",
                        "self": "'<django_prometheus.db.common.ExportingCursorWrapper.<locals>.CursorWrapper [clos'+50",
                        "vendor": "postgresql",
                    },
                    "name": "execute",
                },
                {
                    "filename": "/ak-root/venv/lib/python3.12/site-packages/psycopg/cursor.py",
                    "line": "",
                    "lineno": 732,
                    "locals": {
                        "binary": "None",
                        "params": "'(datetime.datetime(2024, 9, 3, 15, 59, 42, 351305, tzinfo=datetime.timezone.utc)'+2",
                        "prepare": "None",
                        "query": '\'SELECT "authentik_enterprise_licenseusage"."expiring", "authentik_enterprise_lic\'+490',
                        "self": "'<django_prometheus.db.common.ExportingCursorWrapper.<locals>.CursorWrapper [clos'+50",
                    },
                    "name": "execute",
                },
            ],
            "is_cause": true,
            "syntax_error": null,
        },
    ],
    "level": "error",
    "logger": "django.request",
    "timestamp": 1725379182.3523173,
}
BeryJu commented 1 month ago

You can use the image ghcr.io/goauthentik/dev-server:gh-version-2024.8 until 2024.8.1 is released

qwreey commented 1 month ago

For docker-compose:

To fix without changing image, You can get migration file and manually mount it

  1. Get 0004_alter_ldapprovider_options_and_more.py file from PR
curl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/goauthentik/authentik/7637abb1f73187b9d8f199232cb320c1bc6007c3/authentik/providers/ldap/migrations/0004_alter_ldapprovider_options_and_more.py -o 0004_alter_ldapprovider_options_and_more.py
  1. mount fixed migration file 
    - ./0004_alter_ldapprovider_options_and_more.py:/authentik/providers/ldap/migrations/0004_alter_ldapprovider_options_and_more.py

    EX:

    authentik:
image: ghcr.io/goauthentik/server:latest
command: server
volumes:
  - ./authentik/media:/media
  - ./authentik/custom-templates:/templates
  - ./0004_alter_ldapprovider_options_and_more.py:/authentik/providers/ldap/migrations/0004_alter_ldapprovider_options_and_more.py
 ...

Done! enjoy: after migration done, you can delete downloaded file and remove mount image