Upgrade to 2.33.0 failed, missing database migration?

[jas02]

Upgrade of DefectDojo 2.24.1 to 2.33.0 failed We tried to directly upgrade DefectDojo 2.24.1 to DefectDojo 2.33.0 running on Kubernetes, using Helm. Helm chart version 1.6.75 -> 1.6.118.

After the pods come up the instance is broken, with bellow error.

Bug description After the upgrade, pods were started, but UI is broken with error logs below. It looks like the database wasn't migrated. How to fix this?

Steps to reproduce Steps to reproduce the behavior:

1. Change Helm chart values
2. Redeploy helm chart
3. Wait for pods to start
4. Checkout web interface

Expected behavior Upgraded instance with working database model and web interface.

Deployment method (select with an X)

• [ ] Docker Compose
• [x] Kubernetes
• [ ] GoDojo

Environment information

• Kubernetes v1.24.17 (EKS)
• DefectDojo version 2.33.0

Logs

Environment:

Request Method: GET
Request URL: https://defectdojo.<our URL>/dashboard

Django Version: 4.1.13
Python Version: 3.11.4
Installed Applications:
('django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.sites',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'polymorphic',
 'django.contrib.admin',
 'django.contrib.humanize',
 'gunicorn',
 'auditlog',
 'dojo',
 'watson',
 'tagging',
 'imagekit',
 'multiselectfield',
 'rest_framework',
 'rest_framework.authtoken',
 'dbbackup',
 'django_celery_results',
 'social_django',
 'drf_spectacular',
 'drf_spectacular_sidecar',
 'tagulous',
 'fontawesomefree',
 'django_filters',
 'django_prometheus')
Installed Middleware:
['django_prometheus.middleware.PrometheusBeforeMiddleware',
 'django.middleware.common.CommonMiddleware',
 'dojo.middleware.APITrailingSlashMiddleware',
 'dojo.middleware.DojoSytemSettingsMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.middleware.security.SecurityMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'dojo.remote_user.RemoteUserMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'dojo.middleware.LoginRequiredMiddleware',
 'dojo.middleware.AdditionalHeaderMiddleware',
 'social_django.middleware.SocialAuthExceptionMiddleware',
 'watson.middleware.SearchContextMiddleware',
 'auditlog.middleware.AuditlogMiddleware',
 'crum.CurrentRequestUserMiddleware',
 'dojo.request_cache.middleware.RequestCacheMiddleware',
 'django_prometheus.middleware.PrometheusAfterMiddleware']

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django_prometheus/db/common.py", line 69, in execute
    return super().execute(*args, **kwargs)

The above exception (column dojo_finding.sla_expiration_date does not exist
LINE 1: ...col3", "dojo_finding"."sla_start_date" AS "col4", "dojo_find...
                                                             ^
) was the direct cause of the following exception:
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/app/dojo/home/views.py", line 38, in dashboard
    .count()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 621, in count
    return self.query.get_count(using=self.db)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/query.py", line 559, in get_count
    return obj.get_aggregation(using, ["__count"])["__count"]
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/query.py", line 544, in get_aggregation
    result = compiler.execute_sql(SINGLE)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 102, in execute
    return super().execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django_prometheus/db/common.py", line 69, in execute
    return super().execute(*args, **kwargs)

Exception Type: ProgrammingError at /dashboard
Exception Value: column dojo_finding.sla_expiration_date does not exist
LINE 1: ...col3", "dojo_finding"."sla_start_date" AS "col4", "dojo_find...
                                                             ^

[kiblik]

Hi @jas02 Can you

• run python manage.py showmigrations in uwsgi container of django pod?
• check when initializer-job was triggered last time?

[jas02]

OK, I figured out what was wrong. My colleagues has had disabled initializer job in the Helm chart, which was root cause of missing migration (obviously). After enabling this section in Helm chart was the migration and whole upgrade successful.

Closing this incident, thanks @kiblik for help.