wookie184
commented
2 years ago I'll probably create individual issues for these actually
Open wookie184 opened 2 years ago
wookie184
commented
2 years ago I'll probably create individual issues for these actually
There are currently a couple of bugs with our sending of deleted messages to site.
Race condition with antispam:
To reproduce, spam
!clean until <message_id>in a channel until you get the error. Antispam and the clean cog both try and delete the messages and upload them at the same time, causing the error.A potential solution would be to allow two message deletion contexts to share a message on the site side. Not exactly sure how this would work.
Unable to delete webhooks
To reproduce, try and delete a webhook using the clean command. It seems we just don't have handling for this.
Error sending embeds with null bytes
Error
postgres_1 | 2022-05-29 09:56:45.501 UTC [4417] ERROR: unsupported Unicode escape sequence at character 205 postgres_1 | 2022-05-29 09:56:45.501 UTC [4417] DETAIL: \u0000 cannot be converted to text. postgres_1 | 2022-05-29 09:56:45.501 UTC [4417] CONTEXT: JSON data, line 1: {"type": "rich", "title":... postgres_1 | 2022-05-29 09:56:45.501 UTC [4417] STATEMENT: INSERT INTO "api_deletedmessage" ("id", "author_id", "channel_id", "content", "embeds", "attachments", "deletion_context_id") VALUES (980409284938592286, 697133219862151208, 813116922118930459, '', ARRAY['{"type": "rich", "title": "hello\u0000"}']::jsonb[], '{}'::varchar(512)[], 4) web_1 | Internal Server Error: /api/bot/deleted-messages web_1 | Traceback (most recent call last): web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute web_1 | return self.cursor.execute(sql, params) web_1 | psycopg2.errors.UntranslatableCharacter: unsupported Unicode escape sequence web_1 | LINE 1: ...697133219862151208, 813116922118930459, '', ARRAY['{"type": ... web_1 | ^ web_1 | DETAIL: \u0000 cannot be converted to text. web_1 | CONTEXT: JSON data, line 1: {"type": "rich", "title":... web_1 | web_1 | web_1 | The above exception was the direct cause of the following exception: web_1 | web_1 | Traceback (most recent call last): web_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/exception.py", line 47, in inner web_1 | response = get_response(request) web_1 | File "/usr/local/lib/python3.9/site-packages/django/core/handlers/base.py", line 181, in _get_response web_1 | response = wrapped_callback(request, *callback_args, **callback_kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/views.py", line 67, in sentry_wrapped_callback web_1 | return callback(request, *args, **kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view web_1 | return view_func(*args, **kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/viewsets.py", line 125, in view web_1 | return self.dispatch(request, *args, **kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 509, in dispatch web_1 | response = self.handle_exception(exc) web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 469, in handle_exception web_1 | self.raise_uncaught_exception(exc) web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception web_1 | raise exc web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/views.py", line 506, in dispatch web_1 | response = handler(request, *args, **kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/mixins.py", line 19, in create web_1 | self.perform_create(serializer) web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/mixins.py", line 24, in perform_create web_1 | serializer.save() web_1 | File "/usr/local/lib/python3.9/site-packages/rest_framework/serializers.py", line 205, in save web_1 | self.instance = self.create(validated_data) web_1 | File "/app/pydis_site/apps/api/serializers.py", line 126, in create web_1 | DeletedMessage.objects.create( web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method web_1 | return getattr(self.get_queryset(), name)(*args, **kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 447, in create web_1 | obj.save(force_insert=True, using=self.db) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 753, in save web_1 | self.save_base(using=using, force_insert=force_insert, web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 790, in save_base web_1 | updated = self._save_table( web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 895, in _save_table web_1 | results = self._do_insert(cls._base_manager, using, fields, returning_fields, raw) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/base.py", line 933, in _do_insert web_1 | return manager._insert( web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/manager.py", line 85, in manager_method web_1 | return getattr(self.get_queryset(), name)(*args, **kwargs) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/query.py", line 1254, in _insert web_1 | return query.get_compiler(using=using).execute_sql(returning_fields) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/models/sql/compiler.py", line 1397, in execute_sql web_1 | cursor.execute(sql, params) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 98, in execute web_1 | return super().execute(sql, params) web_1 | File "/usr/local/lib/python3.9/site-packages/sentry_sdk/integrations/django/__init__.py", line 493, in execute web_1 | return real_execute(self, sql, params) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 66, in execute web_1 | return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 75, in _execute_with_wrappers web_1 | return executor(sql, params, many, context) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute web_1 | return self.cursor.execute(sql, params) web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/utils.py", line 90, in __exit__ web_1 | raise dj_exc_value.with_traceback(traceback) from exc_value web_1 | File "/usr/local/lib/python3.9/site-packages/django/db/backends/utils.py", line 84, in _execute web_1 | return self.cursor.execute(sql, params) web_1 | django.db.utils.DataError: unsupported Unicode escape sequence web_1 | LINE 1: ...697133219862151208, 813116922118930459, '', ARRAY['{"type": ... web_1 | ^ web_1 | DETAIL: \u0000 cannot be converted to text. web_1 | CONTEXT: JSON data, line 1: {"type": "rich", "title":... web_1 | web_1 | "POST /api/bot/deleted-messages HTTP/1.1" 500 235164 bot_1 | 2022-05-29 09:56:45 | charset_normalizer | TRACE | Detected declarative mark in sequence. Priority +1 given for utf_8. bot_1 | 2022-05-29 09:56:45 | charset_normalizer | TRACE | Code page utf_8 is a multi byte encoding table and it appear that at least one character was encoded using n-bytes. bot_1 | 2022-05-29 09:56:45 | charset_normalizer | TRACE | utf_8 passed initial chaos probing. Mean measured chaos is 0.000000 % bot_1 | 2022-05-29 09:56:45 | charset_normalizer | TRACE | We detected language [('English', 1.0), ('Indonesian', 1.0), ('Simple English', 1.0), ('Dutch', 1.0), ('Swedish', 1.0)] using utf_8 bot_1 | 2022-05-29 09:56:45 | charset_normalizer | DEBUG | Encoding detection: utf_8 is most likely the one. bot_1 | 2022-05-29 09:56:45 | bot.exts.backend.error_handler | WARNING | API responded with 500 for command clean untilTo reproduce, run
And try and clean the resulting message. We currently strip null bytes from message content, but not from any embed contents. https://github.com/python-discord/bot/blob/c29434b964e87743869723da51a7908be7cc7156/bot/exts/moderation/modlog.py#L63
Unknown issue
This is #BOT-34M on sentry. So far we've not managed to work out what the cause of this is or how to reproduce it. #2180 should help us debug this further.