search

DefectDojo / django-DefectDojo

DevSecOps, ASPM, Vulnerability Management. All on one platform.
https://defectdojo.com
BSD 3-Clause "New" or "Revised" License
3.71k stars 1.55k forks source link

Corrupted Postgres #7629

Closed domcar closed 1 year ago

domcar commented 1 year ago

Bug description I think the postgres DB got corrupted while using defectdojo

Steps to reproduce When connected to the UI i get 500 Internal errors for several pages. In the logs of defectdojo I can see errors like: django.db.utils.ProgrammingError: column dojo_jira_project.default_assignee does not exist or django.db.utils.ProgrammingError: column auditlog_logentry.serialized_data does not exist

Deployment method (select with an X)

Environment information I deployed defectDojo using the helm chart version 1.6.43 with the following settings:

postgresql:
  enabled: true
  image:
    tag: 11.16.0-debian-11-r9
  auth:
    username: defectdojo
    password: ""
    database: defectdojo
    existingSecret: defectdojo-postgresql-specific
    secretKeys:
      adminPasswordKey: postgresql-postgres-password
      userPasswordKey: postgresql-password
      replicationPasswordKey: postgresql-replication-password
  architecture: standalone
  primary:
    name: primary
    persistence:
      enabled: true
    affinity: {}
    nodeSelector: {}

Logs 

[pid: 1|app: -|req: -/-] 10.43.79.9 (domcar) {66 vars in 1194 bytes} [Wed Feb 15 07:23:20 2023] GET /product => generated 37217 bytes in 149 msecs (HTTP/1.1 200) 8 headers in 376 bytes (1 switches on core 1)
[15/Feb/2023 07:23:21] INFO [django.request:241] OK: /product
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column dojo_jira_project.default_assignee does not exist
LINE 1: ..."component", "dojo_jira_project"."custom_fields", "dojo_jira...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/product/views.py", line 88, in product
    return render(request, 'dojo/product.html', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/shortcuts.py", line 24, in render
    content = loader.render_to_string(template_name, context, request, using=using)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/backends/django.py", line 62, in render
    return self.template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/defaulttags.py", line 321, in render
    if match:
  File "/usr/local/lib/python3.11/site-packages/django/core/paginator.py", line 177, in __len__
    return len(self.object_list)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 376, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1869, in _fetch_all
    self._prefetch_related_objects()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1258, in _prefetch_related_objects
    prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 2298, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(
                                   ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 2440, in prefetch_one_level
    ) = prefetcher.get_prefetch_queryset(instances, lookup.get_current_queryset(level))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/fields/related_descriptors.py", line 716, in get_prefetch_queryset
    for rel_obj in queryset:
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 394, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: column dojo_jira_project.default_assignee does not exist
LINE 1: ..."component", "dojo_jira_project"."custom_fields", "dojo_jira...
                                                             ^

[pid: 14|app: -|req: -/-] 10.43.79.9 (domcar) {66 vars in 1189 bytes} [Wed Feb 15 07:23:21 2023] GET /product => generated 37217 bytes in 117 msecs (HTTP/1.1 200) 8 headers in 376 bytes (1 switches on core 1)
[pid: 14|app: -|req: -/-] 10.43.79.9 (domcar) {66 vars in 1199 bytes} [Wed Feb 15 07:23:22 2023] GET /product/type => generated 61005 bytes in 104 msecs (HTTP/1.1 200) 8 headers in 376 bytes (1 switches on core 0)
[pid: 14|app: -|req: -/-] 10.43.79.9 (domcar) {66 vars in 1198 bytes} [Wed Feb 15 07:23:23 2023] GET /dashboard => generated 57830 bytes in 178 msecs (HTTP/1.1 200) 8 headers in 376 bytes (1 switches on core 1)
[15/Feb/2023 07:23:25] INFO [django.request:241] OK: /product
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column dojo_jira_project.default_assignee does not exist
LINE 1: ..."component", "dojo_jira_project"."custom_fields", "dojo_jira...
                                                             ^

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/app/dojo/product/views.py", line 88, in product
    return render(request, 'dojo/product.html', {
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/shortcuts.py", line 24, in render
    content = loader.render_to_string(template_name, context, request, using=using)
              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/backends/django.py", line 62, in render
    return self.template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 175, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/loader_tags.py", line 157, in render
    return compiled_parent._render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 167, in _render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/loader_tags.py", line 63, in render
    result = block.nodelist.render(context)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 1005, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/base.py", line 966, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/template/defaulttags.py", line 321, in render
    if match:
  File "/usr/local/lib/python3.11/site-packages/django/core/paginator.py", line 177, in __len__
    return len(self.object_list)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 376, in __len__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1869, in _fetch_all
    self._prefetch_related_objects()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1258, in _prefetch_related_objects
    prefetch_related_objects(self._result_cache, *self._prefetch_related_lookups)
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 2298, in prefetch_related_objects
    obj_list, additional_lookups = prefetch_one_level(
                                   ^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 2440, in prefetch_one_level
    ) = prefetcher.get_prefetch_queryset(instances, lookup.get_current_queryset(level))
        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/fields/related_descriptors.py", line 716, in get_prefetch_queryset
    for rel_obj in queryset:
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 394, in __iter__
    self._fetch_all()
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1867, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
                         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 87, in __iter__
    results = compiler.execute_sql(
              ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1398, in execute_sql
    cursor.execute(sql, params)
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute
    return self._execute_with_wrappers(
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers
    return executor(sql, params, many, context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute
    with self.db.wrap_database_errors:
  File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
django.db.utils.ProgrammingError: column dojo_jira_project.default_assignee does not exist
LINE 1: ..."component", "dojo_jira_project"."custom_fields", "dojo_jira...

This happened suddenly without anything from our side, no upgrade nor manual changes anywhere.

damiencarol commented 1 year ago

Did you checked the initializer container logs?

domcar commented 1 year ago

@damiencarol I can't see any InitContainer in the deployment, are you sure there is one? Or do you refer to something different?

domcar commented 1 year ago

I also tried to modify some settings as test and I get :

django.db.utils.ProgrammingError: column "enable_notify_sla_active" of relation "dojo_system_settings" does not exist
LINE 1: ...ngagement_auto_close_days", "enable_finding_sla", "enable_no...

Something got really wrong with the DB

dsever commented 1 year ago

Try simply exec into DD pod, django container and fro there run ./manage.py showmigrations

It will show you missing migrations

domcar commented 1 year ago

I got the following 

bash-5.1$ ./manage.py showmigrations
[15/Feb/2023 11:24:11] INFO [dojo.models:4168] enabling audit logging
/usr/local/lib/python3.11/site-packages/jira/client.py:11: DeprecationWarning: 'imghdr' is deprecated and slated for removal in Python 3.13
  import imghdr
/app/dojo/reports/views.py:747: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
/app/dojo/engagement/views.py:1218: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
System check identified some issues:

WARNINGS:
?: (staticfiles.W004) The directory '/app/components/node_modules' in the STATICFILES_DIRS setting does not exist.
admin
 [X] 0001_initial
 [X] 0002_logentry_remove_auto_add
 [X] 0003_logentry_add_action_flag_choices
auditlog
 [X] 0001_initial
 [X] 0002_auto_support_long_primary_keys
 [X] 0003_logentry_remote_addr
 [X] 0004_logentry_detailed_object_repr
 [X] 0005_logentry_additional_data_verbose_name
 [X] 0006_object_pk_index
 [X] 0007_object_pk_type
 [X] 0008_action_index
 [X] 0009_alter_logentry_additional_data
 [X] 0010_alter_logentry_timestamp
 [ ] 0011_logentry_serialized_data
 [ ] 0012_add_logentry_action_access
auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
 [X] 0007_alter_validators_add_error_messages
 [X] 0008_alter_user_username_max_length
 [X] 0009_alter_user_last_name_max_length
 [X] 0010_alter_group_name_max_length
 [X] 0011_update_proxy_permissions
 [X] 0012_alter_user_first_name_max_length
authtoken
 [X] 0001_initial
 [X] 0002_auto_20160226_1747
 [X] 0003_tokenproxy
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
django_celery_results
 [X] 0001_initial
 [X] 0002_add_task_name_args_kwargs
 [X] 0003_auto_20181106_1101
 [X] 0004_auto_20190516_0412
 [X] 0005_taskresult_worker
 [X] 0006_taskresult_date_created
 [X] 0007_remove_taskresult_hidden
 [X] 0008_chordcounter
 [X] 0009_groupresult
 [X] 0010_remove_duplicate_indices
 [X] 0011_taskresult_periodic_task_name
dojo
 [X] 0001_squashed_0090_index_duplicate_finding (90 squashed migrations)
 [X] 0091_npm_audit_path_censoring
 [X] 0092_is_mitigated
 [X] 0093_django_tagging_removal
 [X] 0094_remove_system_settings_s_finding_severity_naming
 [X] 0095_remove_old_product_contact_fields
 [X] 0096_grype_name_change
 [X] 0097_engagement_type
 [X] 0098_anchore_vuln_id
 [X] 0099_delete_report
 [X] 0100_dojo_user_for_authv2
 [X] 0101_enable_features
 [X] 0102_dojo_group
 [X] 0103_report_notification
 [X] 0104_endpoint_userinfo_creation
 [X] 0105_endpoint_host_migration
 [X] 0106_role_model
 [X] 0107_global_role
 [X] 0108_blank_fields
 [X] 0109_group_user_role
 [X] 0110_auth_v2_migrate_user_roles
 [X] 0111_group_user_rename
 [X] 0112_group_user_rename_2
 [X] 0113_endpoint_protocol
 [X] 0114_cyclonedx_vuln_uniqu
 [X] 0115_language_types
 [X] 0116_test_type_active
 [X] 0117_usercontactinfo_force_password_reset
 [X] 0118_remove_finding_images
 [X] 0119_default_group_is_staff
 [X] 0120_sonarqube_test_and_clean
 [X] 0121_user_restrict
 [X] 0122_cobaltio_product
 [X] 0123_scan_type
 [X] 0124_sonarqube_api_type_length_change
 [X] 0125_sonarqube_clean
 [X] 0126_finding_publish_date
 [X] 0127_remove_hashes
 [X] 0128_pytz_update
 [X] 0129_finding_deprecated_fields
 [X] 0130_product_api_scan_configuration
 [X] 0131_migrate_sonarcube_cobalt
 [X] 0132_remove_configs_from_test
 [X] 0133_finding_service
 [X] 0134_sonarque_cobaltio_removal
 [X] 0135_email_from
 [X] 0136_default_group_help_text
 [X] 0137_system_settings_enable_endpoint_metadata_import
 [X] 0138_remove_authorized_users
 [X] 0139_google_sheets_rules_framework_enable
 [X] 0140_auth_group
 [X] 0141_enable_user_profile_editable
 [X] 0142_environment_delete
 [X] 0143_objects
 [X] 0144_import_action_untouched
 [X] 0145_system_settings_default_group_email_pattern
 [X] 0146_lead_optional
 [X] 0147_rename_sslyze_parser
 [X] 0148_default_notifications
 [X] 0149_harmonize_user_format
 [X] 0150_dedupe_endpoint_status
 [X] 0151_index_endpoint_status
 [X] 0152_notifications_template
 [X] 0153_migrate_endpoint_mitigated
 [X] 0154_remove_endpoint_mitigated
 [X] 0155_enable_finding_groups
 [X] 0156_migrate_finding_groups_setting
 [X] 0157_vulnerability_reference
 [X] 0158_vulnerability_id
 [X] 0159_remove_broken_endpoint_statuses
 [X] 0160_set_notnull_endpoint_statuses
 [X] 0161_alter_dojo_group_social_provider
 [X] 0162_created_and_updated
 [X] 0163_system_settings_enable_calendar
 [X] 0164_remove_system_settings_staff_user_email_pattern
 [X] 0165_custom_sla
 [X] 0166_copy_sla_from_system_settings
 [X] 0167_system_settings_add_vulnerability_id_to_jira_label
 [X] 0168_alter_system_settings_time_zone
 [X] 0169_planned_remediation_date
 [X] 0170_jira_project_custom_fields
 [X] 0171_jira_labels_per_product_and_engagement
 [X] 0172_optimize_usage_of_endpoint_status
 [X] 0173_alter_risk_acceptance_name
 [ ] 0174_jira_project_default_assignee
 [ ] 0175_system_settings_enable_notify_sla
 [ ] 0176_custom_password_requirements
 [ ] 0177_alter_system_settings_time_zone
 [ ] 0178_alter_answer_polymorphic_ctype_and_more
 [ ] 0179_alter_finding_verified
 [ ] 0180_announcement_userannouncement
sessions
 [X] 0001_initial
sites
 [X] 0001_initial
 [X] 0002_alter_domain_unique
social_django
 [X] 0001_initial (2 squashed migrations)
 [X] 0002_add_related_name (2 squashed migrations)
 [X] 0003_alter_email_max_length (2 squashed migrations)
 [X] 0004_auto_20160423_0400 (2 squashed migrations)
 [X] 0005_auto_20160727_2333 (1 squashed migrations)
 [X] 0006_partial
 [X] 0007_code_timestamp
 [X] 0008_partial_timestamp
 [X] 0009_auto_20191118_0520
 [X] 0010_uid_db_index
tagging
 [X] 0001_initial
 [X] 0002_on_delete
 [X] 0003_adapt_max_tag_length
watson
 [X] 0001_initial
 [ ] 0002_alter_searchentry_object_id

Could you please tell me how to preceed? Do I need to perform migrations?

domcar commented 1 year ago

ok, i did the migration with ./manage.py migrate now they are all shown as completed in the list from showmigrations. Unfortunately this didn't change much, I still get the whole 500 errors in the UI and the same logs as abode. Anything I can do?

dsever commented 1 year ago

Can you point me to the exact logs/error you have now after migration has been applied?

domcar commented 1 year ago

An example is when in the UI I click on the menu --> all Products. This generate in the UI a 500 Internal Server Error and in the logs I see 

[15/Feb/2023 12:31:30] INFO [django.request:241] OK: /product
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedColumn: column dojo_jira_project.default_assignee does not exist
LINE 1: ..."component", "dojo_jira_project"."custom_fields", "dojo_jira...

Or if in the UI I go to menu settings --> announcement the error is

[15/Feb/2023 12:35:25] INFO [django.request:241] OK: /configure_announcement
Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedTable: relation "dojo_announcement" does not exist
LINE 1: ..."."dismissable", "dojo_announcement"."style" FROM "dojo_anno...
                                                             ^
dsever commented 1 year ago

Do you really have [ ] 0180_announcement_userannouncement and [ ] 0174_jira_project_default_assignee checked when using show migration?

domcar commented 1 year ago

I ran again the command just to be sure

bash-5.1$ ./manage.py showmigrations
[15/Feb/2023 14:24:44] INFO [dojo.models:4168] enabling audit logging
/usr/local/lib/python3.11/site-packages/jira/client.py:11: DeprecationWarning: 'imghdr' is deprecated and slated for removal in Python 3.13
  import imghdr
/app/dojo/reports/views.py:747: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
/app/dojo/engagement/views.py:1218: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
System check identified some issues:

WARNINGS:
?: (staticfiles.W004) The directory '/app/components/node_modules' in the STATICFILES_DIRS setting does not exist.
admin
 [X] 0001_initial
 [X] 0002_logentry_remove_auto_add
 [X] 0003_logentry_add_action_flag_choices
auditlog
 [X] 0001_initial
 [X] 0002_auto_support_long_primary_keys
 [X] 0003_logentry_remote_addr
 [X] 0004_logentry_detailed_object_repr
 [X] 0005_logentry_additional_data_verbose_name
 [X] 0006_object_pk_index
 [X] 0007_object_pk_type
 [X] 0008_action_index
 [X] 0009_alter_logentry_additional_data
 [X] 0010_alter_logentry_timestamp
 [X] 0011_logentry_serialized_data
 [X] 0012_add_logentry_action_access
auth
 [X] 0001_initial
 [X] 0002_alter_permission_name_max_length
 [X] 0003_alter_user_email_max_length
 [X] 0004_alter_user_username_opts
 [X] 0005_alter_user_last_login_null
 [X] 0006_require_contenttypes_0002
 [X] 0007_alter_validators_add_error_messages
 [X] 0008_alter_user_username_max_length
 [X] 0009_alter_user_last_name_max_length
 [X] 0010_alter_group_name_max_length
 [X] 0011_update_proxy_permissions
 [X] 0012_alter_user_first_name_max_length
authtoken
 [X] 0001_initial
 [X] 0002_auto_20160226_1747
 [X] 0003_tokenproxy
contenttypes
 [X] 0001_initial
 [X] 0002_remove_content_type_name
django_celery_results
 [X] 0001_initial
 [X] 0002_add_task_name_args_kwargs
 [X] 0003_auto_20181106_1101
 [X] 0004_auto_20190516_0412
 [X] 0005_taskresult_worker
 [X] 0006_taskresult_date_created
 [X] 0007_remove_taskresult_hidden
 [X] 0008_chordcounter
 [X] 0009_groupresult
 [X] 0010_remove_duplicate_indices
 [X] 0011_taskresult_periodic_task_name
dojo
 [X] 0001_squashed_0090_index_duplicate_finding (90 squashed migrations)
 [X] 0091_npm_audit_path_censoring
 [X] 0092_is_mitigated
 [X] 0093_django_tagging_removal
 [X] 0094_remove_system_settings_s_finding_severity_naming
 [X] 0095_remove_old_product_contact_fields
 [X] 0096_grype_name_change
 [X] 0097_engagement_type
 [X] 0098_anchore_vuln_id
 [X] 0099_delete_report
 [X] 0100_dojo_user_for_authv2
 [X] 0101_enable_features
 [X] 0102_dojo_group
 [X] 0103_report_notification
 [X] 0104_endpoint_userinfo_creation
 [X] 0105_endpoint_host_migration
 [X] 0106_role_model
 [X] 0107_global_role
 [X] 0108_blank_fields
 [X] 0109_group_user_role
 [X] 0110_auth_v2_migrate_user_roles
 [X] 0111_group_user_rename
 [X] 0112_group_user_rename_2
 [X] 0113_endpoint_protocol
 [X] 0114_cyclonedx_vuln_uniqu
 [X] 0115_language_types
 [X] 0116_test_type_active
 [X] 0117_usercontactinfo_force_password_reset
 [X] 0118_remove_finding_images
 [X] 0119_default_group_is_staff
 [X] 0120_sonarqube_test_and_clean
 [X] 0121_user_restrict
 [X] 0122_cobaltio_product
 [X] 0123_scan_type
 [X] 0124_sonarqube_api_type_length_change
 [X] 0125_sonarqube_clean
 [X] 0126_finding_publish_date
 [X] 0127_remove_hashes
 [X] 0128_pytz_update
 [X] 0129_finding_deprecated_fields
 [X] 0130_product_api_scan_configuration
 [X] 0131_migrate_sonarcube_cobalt
 [X] 0132_remove_configs_from_test
 [X] 0133_finding_service
 [X] 0134_sonarque_cobaltio_removal
 [X] 0135_email_from
 [X] 0136_default_group_help_text
 [X] 0137_system_settings_enable_endpoint_metadata_import
 [X] 0138_remove_authorized_users
 [X] 0139_google_sheets_rules_framework_enable
 [X] 0140_auth_group
 [X] 0141_enable_user_profile_editable
 [X] 0142_environment_delete
 [X] 0143_objects
 [X] 0144_import_action_untouched
 [X] 0145_system_settings_default_group_email_pattern
 [X] 0146_lead_optional
 [X] 0147_rename_sslyze_parser
 [X] 0148_default_notifications
 [X] 0149_harmonize_user_format
 [X] 0150_dedupe_endpoint_status
 [X] 0151_index_endpoint_status
 [X] 0152_notifications_template
 [X] 0153_migrate_endpoint_mitigated
 [X] 0154_remove_endpoint_mitigated
 [X] 0155_enable_finding_groups
 [X] 0156_migrate_finding_groups_setting
 [X] 0157_vulnerability_reference
 [X] 0158_vulnerability_id
 [X] 0159_remove_broken_endpoint_statuses
 [X] 0160_set_notnull_endpoint_statuses
 [X] 0161_alter_dojo_group_social_provider
 [X] 0162_created_and_updated
 [X] 0163_system_settings_enable_calendar
 [X] 0164_remove_system_settings_staff_user_email_pattern
 [X] 0165_custom_sla
 [X] 0166_copy_sla_from_system_settings
 [X] 0167_system_settings_add_vulnerability_id_to_jira_label
 [X] 0168_alter_system_settings_time_zone
 [X] 0169_planned_remediation_date
 [X] 0170_jira_project_custom_fields
 [X] 0171_jira_labels_per_product_and_engagement
 [X] 0172_optimize_usage_of_endpoint_status
 [X] 0173_alter_risk_acceptance_name
 [X] 0174_jira_project_default_assignee
 [X] 0175_system_settings_enable_notify_sla
 [X] 0176_custom_password_requirements
 [X] 0177_alter_system_settings_time_zone
 [X] 0178_alter_answer_polymorphic_ctype_and_more
 [X] 0179_alter_finding_verified
 [X] 0180_announcement_userannouncement
sessions
 [X] 0001_initial
sites
 [X] 0001_initial
 [X] 0002_alter_domain_unique
social_django
 [X] 0001_initial (2 squashed migrations)
 [X] 0002_add_related_name (2 squashed migrations)
 [X] 0003_alter_email_max_length (2 squashed migrations)
 [X] 0004_auto_20160423_0400 (2 squashed migrations)
 [X] 0005_auto_20160727_2333 (1 squashed migrations)
 [X] 0006_partial
 [X] 0007_code_timestamp
 [X] 0008_partial_timestamp
 [X] 0009_auto_20191118_0520
 [X] 0010_uid_db_index
tagging
 [X] 0001_initial
 [X] 0002_on_delete
 [X] 0003_adapt_max_tag_length
watson
 [X] 0001_initial
 [X] 0002_alter_searchentry_object_id

is there a way maybe to force the migration to run again?

dsever commented 1 year ago

You can run this one [X] 0173_alter_risk_acceptance_name will unapply migrations, and then run one by one migration stating from 0175_system_settings_enable_notify_sla to see if there are any errors

domcar commented 1 year ago

Can you tell me which command to use? Sorry, i'm not familiar with the tool

dsever commented 1 year ago

./manege.py 0173_alter_risk_acceptance_name to rollback

Please take a look at the https://docs.djangoproject.com/en/4.1/ref/django-admin/#migrate for more detail about the other options

domcar commented 1 year ago

Hi @dsever when I try ./manage.py migrate dojo 0173_alter_risk_acceptance_name it ends with the following error:

[17/Feb/2023 07:05:04] INFO [dojo.models:4168] enabling audit logging
/usr/local/lib/python3.11/site-packages/jira/client.py:11: DeprecationWarning: 'imghdr' is deprecated and slated for removal in Python 3.13
  import imghdr
/app/dojo/reports/views.py:747: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
/app/dojo/engagement/views.py:1218: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
System check identified some issues:

WARNINGS:
?: (staticfiles.W004) The directory '/app/components/node_modules' in the STATICFILES_DIRS setting does not exist.
Operations to perform:
  Target specific migration: 0173_alter_risk_acceptance_name, from dojo
Running migrations:
  Rendering model states... DONE
  Unapplying dojo.0180_announcement_userannouncement...Traceback (most recent call last):
  File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute
    return self.cursor.execute(sql, params)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
psycopg2.errors.UndefinedTable: table "dojo_userannouncement" does not exist

So I tried with some previous migration, for example ./manage.py migrate dojo 0139_google_sheets_rules_framework_enable

but it still has errors

[17/Feb/2023 07:07:49] INFO [dojo.models:4168] enabling audit logging
/usr/local/lib/python3.11/site-packages/jira/client.py:11: DeprecationWarning: 'imghdr' is deprecated and slated for removal in Python 3.13
  import imghdr
/app/dojo/reports/views.py:747: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
/app/dojo/engagement/views.py:1218: DeprecationWarning: invalid escape sequence '\?'
  path_items = list(filter(None, re.split('/|\?', url))) # noqa W605
System check identified some issues:

WARNINGS:
?: (staticfiles.W004) The directory '/app/components/node_modules' in the STATICFILES_DIRS setting does not exist.
Operations to perform:
  Target specific migration: 0139_google_sheets_rules_framework_enable, from dojo
Running migrations:
  Rendering model states...Killed

Do you have other ideas what I could try?

domcar commented 1 year ago

Ehi, I created manually those tables and it worked a bit more, but now I get django.db.migrations.exceptions.IrreversibleError: Operation <RunPython <function migrate_notify_sla_from_settings_file at 0x7f4855c31e40>> in dojo.0175_system_settings_enable_notify_sla is not reversible when running the migration

dsever commented 1 year ago

Nothing should be created manually... do you have backup maybe?

domcar commented 1 year ago

I restored the backup and everything is going fine. I still would like to know what happened, to avoid it happening in the future, do you have any idea how I could find this out?

dsever commented 1 year ago

To be honest, have no idea hot to replicate it. Can we close this issue?