Semgrep parser: Bug in uploading scan results with emojii

[shizzgar]

I have such log info from my running Dojo instance.

uwsgi_1 | [24/Feb/2022 13:21:52] ERROR [dojo.api_v2.exception_handler:32] (1366, "Incorrect string value: '\xF0\x9F\x98\x8E ...' for column 'description' at row 1") uwsgi_1 | Traceback (most recent call last): uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute uwsgi_1 | return self.cursor.execute(sql, params) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute uwsgi_1 | return self.cursor.execute(query, args) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute uwsgi_1 | res = self._query(query) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query uwsgi_1 | db.query(q) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query uwsgi_1 | _mysql.connection.query(self, query) uwsgi_1 | MySQLdb._exceptions.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x98\x8E ...' for column 'description' at row 1") uwsgi_1 | uwsgi_1 | The above exception was the direct cause of the following exception: uwsgi_1 | uwsgi_1 | Traceback (most recent call last): uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/rest_framework/views.py", line 506, in dispatch uwsgi_1 | response = handler(request, *args, kwargs) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/rest_framework/mixins.py", line 19, in create uwsgi_1 | self.perform_create(serializer) uwsgi_1 | File "/app/./dojo/api_v2/views.py", line 2075, in perform_create uwsgi_1 | serializer.save(push_to_jira=push_to_jira) uwsgi_1 | File "/app/./dojo/api_v2/serializers.py", line 1465, in save uwsgi_1 | reimporter.reimport_scan(scan, scan_type, test, active=active, verified=verified, uwsgi_1 | File "/app/./dojo/importers/reimporter/reimporter.py", line 333, in reimport_scan uwsgi_1 | self.process_parsed_findings(test, parsed_findings, scan_type, user, active, verified, uwsgi_1 | File "/app/./dojo/decorators.py", line 44, in wrapper uwsgi_1 | return func(*args, *kwargs) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/celery/local.py", line 188, in call uwsgi_1 | return self._get_current_object()(a, kw) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/celery/app/task.py", line 389, in call uwsgi_1 | return self.run(*args, kwargs) uwsgi_1 | File "/app/./dojo/importers/reimporter/reimporter.py", line 148, in process_parsed_findings uwsgi_1 | item.save(dedupe_option=False) uwsgi_1 | File "/app/./dojo/models.py", line 2377, in save uwsgi_1 | super(Finding, self).save(*args, *kwargs) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 739, in save uwsgi_1 | self.save_base(using=using, force_insert=force_insert, uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 776, in save_base uwsgi_1 | updated = self._save_table( uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 881, in _save_table uwsgi_1 | results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/base.py", line 919, in _do_insert uwsgi_1 | return manager._insert( uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/manager.py", line 85, in manager_method uwsgi_1 | return getattr(self.get_queryset(), name)(args, kwargs) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/query.py", line 1270, in _insert uwsgi_1 | return query.get_compiler(using=using).execute_sql(returning_fields) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/models/sql/compiler.py", line 1416, in execute_sql uwsgi_1 | cursor.execute(sql, params) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 66, in execute uwsgi_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers uwsgi_1 | return executor(sql, params, many, context) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute uwsgi_1 | return self.cursor.execute(sql, params) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/utils.py", line 90, in exit uwsgi_1 | raise dj_exc_value.with_traceback(traceback) from exc_value uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/utils.py", line 84, in _execute uwsgi_1 | return self.cursor.execute(sql, params) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/django/db/backends/mysql/base.py", line 73, in execute uwsgi_1 | return self.cursor.execute(query, args) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 206, in execute uwsgi_1 | res = self._query(query) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/MySQLdb/cursors.py", line 319, in _query uwsgi_1 | db.query(q) uwsgi_1 | File "/usr/local/lib/python3.8/site-packages/MySQLdb/connections.py", line 254, in query uwsgi_1 | _mysql.connection.query(self, query) uwsgi_1 | django.db.utils.OperationalError: (1366, "Incorrect string value: '\xF0\x9F\x98\x8E *...' for column 'description' at row 1") uwsgi_1 | [24/Feb/2022 13:21:52] ERROR [django.request:224] Internal Server Error: /api/v2/reimport-scan/ uwsgi_1 | ERROR:django.request:Internal Server Error: /api/v2/reimport-scan/

Semgrep scan result: semgrep_dojo_upload_brake.json.txt

Bug description

Bug in semgrep upload (but i think in every scanner)

Dojo (or mysql) can't process results from scanners, where emoji with smile_with_sunglasses (this one -> \xF0\x9F\x98\x8E -> 😎) appears.

Deployment method (select with an X)

• [X ] Docker Compose
• [ ] Kubernetes
• [ ] GoDojo

[valentijnscholten]

please check the existing issue https://github.com/DefectDojo/django-DefectDojo/issues/5912

[manuel-sommer]

I have tested the uploaded result example with the latest semgrep. It works fine. You can close this @mtesauro