Initializer not triggered in terraform deployment

[Halogenmake]

Bug description Good day! I'm trying to install DefectDojo on my k8s cluster. We use Terragrunt for the installation (essentially the same as Helm). During the installation process, the defectdojo-django-*- pod does not deploy. In the uWSGI image, the following error occurs: django.db.utils.ProgrammingError: (1146, "Table 'defectdojo.django_site' doesn't exist").

For my installation, I use a MySQL database. When I connect to the database locally, I indeed do not see this table.

I am installing the latest release from the master branch. The version of the HELM chart is 1.6.129. Can you advise how to solve this problem?

The installation is performed with the following parameters, different from the default settings:

host: defectdojo.mysite.net
createSecret: true
createMysqlSecret: true
createRedisSecret: true
createPostgresqlSecret: true

database: mysql

postgresql:
  enabled: false

mysql:
  enabled: true

celery:
  broker: redis

rabbitmq:
  enabled: false

redis:
  enabled: true

django:
  ingress:
    activateTLS: false
    enabled: false
    secretName: ""

mediaPersistentVolume:
  enabled: true
  # any name
  name: media
  # could be emptyDir (not for production) or pvc
  type: pvc
  persistentVolumeClaim:
    create: true
    name:
    size: 5Gi
    accessModes:
      - ReadWriteMany
    storageClassName:

Deployment method (select with an X)

• [ ] Docker Compose
• [X] Kubernetes
• [ ] GoDojo

Environment information

• Operating System: k8s
• DefectDojo version latest

Logs

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 56, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/decorators.py", line 186, in _wrapped
    return fn(request, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/decorators.py", line 186, in _wrapped
    return fn(request, *args, **kw)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/user/views.py", line 153, in login_view
    return DojoLoginView.as_view(template_name='dojo/login.html', authentication_form=AuthenticationForm)(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 103, in view
    return self.dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/decorators/debug.py", line 92, in sensitive_post_parameters_wrapper
    return view(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 134, in _wrapped_view
    response = view_func(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/utils/decorators.py", line 46, in _wrapper
    return bound_method(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/decorators/cache.py", line 62, in _wrapped_view_func
    response = view_func(request, *args, **kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/views.py", line 90, in dispatch
    return super().dispatch(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/base.py", line 142, in dispatch
    return handler(request, *args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/views/generic/edit.py", line 144, in get
    return self.render_to_response(self.get_context_data())
                                   ^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/auth/views.py", line 114, in get_context_data
    current_site = get_current_site(self.request)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/sites/shortcuts.py", line 16, in get_current_site
    return Site.objects.get_current(request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/sites/models.py", line 59, in get_current
    return self._get_site_by_id(site_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/contrib/sites/models.py", line 30, in _get_site_by_id
    site = self.get(pk=site_id)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 85, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 646, in get
    num = len(clone)
          ^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 376, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/mysql/base.py", line 75, in execute
    return self.cursor.execute(query, args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/MySQLdb/cursors.py", line 206, in execute
    res = self._query(query)

          ^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/MySQLdb/cursors.py", line 319, in _query
    db.query(q)
  File "/usr/local/lib/python3.11/site-packages/MySQLdb/connections.py", line 254, in query
    _mysql.connection.query(self, query)
django.db.utils.ProgrammingError: (1146, "Table 'defectdojo.django_site' doesn't exist")
[pid: 19|app: -|req: -/-] 10.100.10.1 (-) {34 vars in 416 bytes} [Mon May 13 19:18:44 2024] GET /uwsgi_health => generated 12833 bytes in 35 msecs (HTTP/1.1 500) 7 headers in 242 bytes (1 switches on core 1)

Screenshots

[kiblik]

Hi @Halogenmake. Thanks for the question.

1. I highly recommend reconsidering the usage of MySQL. It will not be supported soon: https://github.com/DefectDojo/django-DefectDojo/discussions/9690
2. Can you double-check the health of the following job? https://github.com/DefectDojo/django-DefectDojo/blob/master/helm/defectdojo/templates/initializer-job.yaml. It is responsible for DB preparation. If it failed, check why.

[Halogenmake]

@kiblik Thank you for your response! I don't see the job being launched. It's not in the namespace where I'm trying to install DefectDojo (namespace: defectdojo) nor in the default.

[kiblik]

There is ttlSecondsAfterFinished for this job so there is a chance that it was already removed. Can you trigger an upgrade of the chart and check how is initializer behaving? If it is not triggered, double-check that .Values.initializer.run is true.

[Halogenmake]

It seems that the initializer is not triggering and not starting, even though its parameter is set to true.

initializer:
  run: true
  jobAnnotations: {helm.sh/hook: "post-install,post-upgrade"}
  annotations: {}
  labels: {}
  keepSeconds: 60
  affinity: {}
  nodeSelector: {}
  resources:
    requests:
      cpu: 100m
      memory: 256Mi
    limits:
      cpu: 2000m
      memory: 512Mi

[kiblik]

I can only recommend running helm template --debug/helm install --dry-run --debug/--dry-run=server, check if initializer is defined there and dig deeper there. I do not have any other recommendation here.

But please provide feedback if you will find out what was the reason. I'm quite curious.

[Halogenmake]

@kiblik It's a very strange situation. If I install using Helm from the downloaded repository, the job starts and everything is fine, but if I install through Terragrunt, the job refuses to start. When installing through Terragrunt, I specify the following repository:

resource "helm_release" "defectdojo" {
  name             = "defectdojo"
  repository       = "https://raw.githubusercontent.com/DefectDojo/django-DefectDojo/helm-charts"
  chart            = "defectdojo"
  namespace        = var.namespace
  create_namespace = true
  timeout          = 600
  version          = var.chart_version
  values = [
    var.extra_values
  ]

[kiblik]

@kiblik It's a very strange situation. If I install using Helm from the downloaded repository, the job starts and everything is fine, but if I install through Terragrunt, the job refuses to start. When installing through Terragrunt, I specify the following repository:

resource "helm_release" "defectdojo" {
  name             = "defectdojo"
  repository       = "https://raw.githubusercontent.com/DefectDojo/django-DefectDojo/helm-charts"
  chart            = "defectdojo"
  namespace        = var.namespace
  create_namespace = true
  timeout          = 600
  version          = var.chart_version
  values = [
    var.extra_values
  ]

@Halogenmake, I have zero experience with Terragrunt. I'm not able to help you here. If you would find a fix, feel free to share.

[HaziFlorinMarian]

I have the same issue and it's quite strange that helm command deploys initializer container but terraform helm_Release resource doesn't. Both manual command and terraform use same values.yaml with initializer enabled.

If someone else will have same issue with terraform just add the following parameter to the helm_release resource to fix initializer:

  wait = false # Fix initializer

[kiblik]

Can we rename this issue? It is still not solved but we identified that the issue is with "not triggered initializer in terraform deployment". Or maybe even close it because the issue is outside of DefectDojo.