Maffooch
commented
7 months ago I tried reproducing this a few ways, but did not experience any issues:
- Load OpenVAS CSV/XML and Clair/ClairKlar findings into an instance of 2.30.4 and then upgrade to 2.31.2
- Upgrade from 2.30.4 to 2.31.2 without OpenVAS CSV/XML and Clair/ClairKlar findings
The only way I could reproduce this one is to have a Clair Scan test type set to inactive, but still exist in the database. Is that what your configuration looks like? If so, setting Clair Scan to active fixed this issue for me.
37b
kiblik
Bug description When installing
2.31.0, or2.31.1from version2.30.4the migrations fail and the app throws errors in UI anduwsgi.Steps to reproduce Steps to reproduce the behavior:
2.31.0or2.31.1.Expected behavior I expect the migrations to complete
Deployment method (select with an
X)Environment information
2.30.0or2.31.0Logs
Running migrations: [21/Feb/2024 00:49:06] WARNING [dojo.db_migrations.0197_parser_merge:62] We identified 0 OpenVAS CSV/ OpenVAS XML findings to migrate to OpenVAS Parser findings Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 929, in get_or_create return self.get(**kwargs), False ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 650, in get raise self.model.DoesNotExist( __fake__.Test_Type.DoesNotExist: Test_Type matching query does not exist. During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django_prometheus/db/common.py", line 69, in execute return super().execute(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/MySQLdb/cursors.py", line 206, in execute res = self._query(query) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/MySQLdb/cursors.py", line 319, in _query db.query(q) File "/usr/local/lib/python3.11/site-packages/MySQLdb/connections.py", line 254, in query _mysql.connection.query(self, query) MySQLdb.IntegrityError: (1062, "Duplicate entry 'Clair Scan' for key 'dojo_test_type.name'") The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/app/manage.py", line 11, in <module> execute_from_command_line(sys.argv) File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 446, in execute_from_command_line utility.execute() File "/usr/local/lib/python3.11/site-packages/django/core/management/__init__.py", line 440, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 402, in run_from_argv self.execute(*args, **cmd_options) File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 448, in execute output = self.handle(*args, **options) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/core/management/base.py", line 96, in wrapped res = handle_func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/core/management/commands/migrate.py", line 349, in handle post_migrate_state = executor.migrate( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 135, in migrate state = self._migrate_all_forwards( ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 167, in _migrate_all_forwards state = self.apply_migration( ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/migrations/executor.py", line 252, in apply_migration state = migration.apply(state, schema_editor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/migrations/migration.py", line 125, in apply operation.database_forwards( File "/usr/local/lib/python3.11/site-packages/django/db/migrations/operations/special.py", line 193, in database_forwards self.code(from_state.apps, schema_editor) File "/app/dojo/db_migrations/0197_parser_merge.py", line 76, in migrate_clairklar_parsers clair_test_type, _ = test_type_model.objects.get_or_create(name="Clair Scan", active=True) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 936, in get_or_create return self.create(**params), True ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 671, in create obj.save(force_insert=True, using=self.db) File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 812, in save self.save_base( File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 863, in save_base updated = self._save_table( ^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1006, in _save_table results = self._do_insert( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/base.py", line 1047, in _do_insert return manager._insert( ^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/manager.py", line 85, in manager_method return getattr(self.get_queryset(), name)(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/query.py", line 1791, in _insert return query.get_compiler(using=using).execute_sql(returning_fields) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/models/sql/compiler.py", line 1660, in execute_sql cursor.execute(sql, params) File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 67, in execute return self._execute_with_wrappers( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 80, in _execute_with_wrappers return executor(sql, params, many, context) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 84, in _execute with self.db.wrap_database_errors: File "/usr/local/lib/python3.11/site-packages/django/db/utils.py", line 91, in __exit__ raise dj_exc_value.with_traceback(traceback) from exc_value File "/usr/local/lib/python3.11/site-packages/django/db/backends/utils.py", line 89, in _execute return self.cursor.execute(sql, params) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django_prometheus/db/common.py", line 69, in execute return super().execute(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/django/db/backends/mysql/base.py", line 75, in execute return self.cursor.execute(query, args) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/MySQLdb/cursors.py", line 206, in execute res = self._query(query) ^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/MySQLdb/cursors.py", line 319, in _query db.query(q) File "/usr/local/lib/python3.11/site-packages/MySQLdb/connections.py", line 254, in query _mysql.connection.query(self, query) django.db.utils.IntegrityError: (1062, "Duplicate entry 'Clair Scan' for key 'dojo_test_type.name'") Applying dojo.0197_parser_merge...Admin user: admin /app/dojo/models.py:4343: ManagerInheritanceWarning: PolymorphicModel: "Question.objects" manager is of type "Manager", but must be a subclass of PolymorphicManager. to support retrieving subclasses class Question(PolymorphicModel, TimeStampedModel): /app/dojo/models.py:4474: ManagerInheritanceWarning: PolymorphicModel: "Answer.objects" manager is of type "Manager", but must be a subclass of PolymorphicManager. to support retrieving subclasses class Answer(PolymorphicModel, TimeStampedModel): Admin password: Initialization detected that the admin user admin already exists in your database. If you don't remember the admin password, you can create a new superuser with: $ docker compose exec uwsgi /bin/bash -c 'python manage.py createsuperuser' Initialization of test_types /app/dojo/models.py:4343: ManagerInheritanceWarning: PolymorphicModel: "Question.objects" manager is of type "Manager", but must be a subclass of PolymorphicManager. to support retrieving subclasses class Question(PolymorphicModel, TimeStampedModel): /app/dojo/models.py:4474: ManagerInheritanceWarning: PolymorphicModel: "Answer.objects" manager is of type "Manager", but must be a subclass of PolymorphicManager. to support retrieving subclasses class Answer(PolymorphicModel, TimeStampedModel): /usr/local/lib/python3.11/site-packages/polymorphic/base.py:197: ManagerInheritanceWarning: PolymorphicModelBase._default_manager is not a PolymorphicManager warnings.warn( /usr/local/lib/python3.11/site-packages/coreapi/codecs/download.py:5: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13 import cgi /usr/local/lib/python3.11/site-packages/polymorphic/base.py:197: ManagerInheritanceWarning: PolymorphicModelBase._default_manager is not a PolymorphicManager warnings.warn( Creation of non-standard permissions /app/dojo/models.py:4343: ManagerInheritanceWarning: PolymorphicModel: "Question.objects" manager is of type "Manager", but must be a subclass of PolymorphicManager. to support retrieving subclasses class Question(PolymorphicModel, TimeStampedModel): /app/dojo/models.py:4474: ManagerInheritanceWarning: PolymorphicModel: "Answer.objects" manager is of type "Manager", but must be a subclass of PolymorphicManager. to support retrieving subclasses class Answer(PolymorphicModel, TimeStampedModel): /usr/local/lib/python3.11/site-packages/polymorphic/base.py:197: ManagerInheritanceWarning: PolymorphicModelBase._default_manager is not a PolymorphicManager warnings.warn( /usr/local/lib/python3.11/site-packages/coreapi/codecs/download.py:5: DeprecationWarning: 'cgi' is deprecated and slated for removal in Python 3.13 import cgi /usr/local/lib/python3.11/site-packages/polymorphic/base.py:197: ManagerInheritanceWarning: PolymorphicModelBase._default_manager is not a PolymorphicManager warnings.warn( [21/Feb/2024 00:49:14] INFO [dojo.management.commands.initialize_permissions:28] Non-standard permissions have been createdSample scan files None
Screenshots N/A
Additional context (optional) Using Aurora RDS with MySQL 8.0 compatibility.