search

ephios-dev / ephios

ephios is a django web application for managing participation for services at events, like paramedics at a festival, lifeguards at the beach, referees and judges at competitions, etc.
https://ephios.de
MIT License
26 stars 8 forks source link

Unsupported characters lead to errors #1209

Closed pov91 closed 7 months ago

pov91 commented 8 months ago

If users copy messages from chats into the description of events and these contain smileys that cannot be displayed by ephios despite the character set utf8mb4, an error occurs.

Subject of the error emails: [Django] Notification sending failed [Django] ERROR (EXTERNAL IP): Internal Server Error: /events/540/edit/

Here it would make sense if ephios filtered out characters that it cannot process so that this user behavior does not lead to repeated errors.

The error can be solved individually by changing the description so that the smileys are no longer present and all notifications under https://your-url.de/admin/core/notification/ are deleted.

felixrindt commented 8 months ago

Can you post some more of the error message? Like some specific exception name and location?

We are using the standard Django char fields which should support all of unicode with all the text sanitation in place to not cause errors on server or client side.

Please check the Django docs to make sure your database is configured correctly to work with all kinds of unicode data.

Also if you happen to identify the character/codepoint that is causing trouble, please tell us so we can recreate the issue.

pov91 commented 8 months ago

First error message "[Django] ERROR (EXTERNAL IP): Internal Server Error: /events/540/edit/":

Internal Server Error: /events/540/edit/

OperationalError at /events/540/edit/
(1366, "Incorrect string value: '\\xF0\\x9F\\xAA\\xA2\\x0D\\x0A...' for column `ephiosdb`.`event`.`description` at row 1")

Request Method: POST
Request URL: *******/events/540/edit/
Django Version: 5.0.2
Python Executable: /home/ephios/venv/bin/python3
Python Version: 3.10.12
Python Path: ['/home/ephios', '/home/ephios/venv/bin', '/usr/lib/python310.zip', '/usr/lib/python3.10', '/usr/lib/python3.10/lib-dynload', '/home/ephios/venv/lib/python3.10/site-packages']
Server time: Tue, 20 Feb 2024 08:10:49 +0100
Installed Applications:
['ephios.core',
 'ephios.extra',
 'ephios.api',
 'django.contrib.admin',
 'django.contrib.auth',
 'polymorphic',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.messages',
 'django.contrib.staticfiles',
 'django.contrib.humanize',
 'django_filters',
 'guardian',
 'oauth2_provider',
 'rest_framework',
 'django_select2',
 'djangoformsetjs',
 'compressor',
 'recurrence',
 'statici18n',
 'dynamic_preferences.users.apps.UserPreferencesConfig',
 'crispy_forms',
 'crispy_bootstrap5',
 'webpush',
 'ephios.modellogging',
 'ephios.plugins.basesignup.apps.PluginApp',
 'ephios.plugins.complexsignup.apps.PluginApp',
 'ephios.plugins.pages.apps.PluginApp',
 'ephios.plugins.qualification_management.apps.PluginApp',
 'ephios.plugins.guests.apps.PluginApp',
 'ephios.plugins.eventautoqualification.apps.PluginApp',
 'ephios.plugins.simpleresource.apps.PluginApp',
 'ephios.plugins.federation.apps.PluginApp',
 'dynamic_preferences']
Installed Middleware:
['django.middleware.security.SecurityMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.middleware.locale.LocaleMiddleware',
 'ephios.extra.middleware.EphiosLocaleMiddleware',
 'ephios.extra.middleware.EphiosNotificationMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.middleware.csrf.CsrfViewMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware',
 'django.contrib.messages.middleware.MessageMiddleware',
 'django.middleware.clickjacking.XFrameOptionsMiddleware',
 'csp.middleware.CSPMiddleware',
 'ephios.modellogging.middleware.LoggingRequestMiddleware']

Traceback (most recent call last):
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 76, in execute
    return self.cursor.execute(query, args)
  File "/home/ephios/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
  File "/home/ephios/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/home/ephios/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)

The above exception ((1366, "Incorrect string value: '\\xF0\\x9F\\xAA\\xA2\\x0D\\x0A...' for column `ephiosdb`.`event`.`description` at row 1")) was the direct cause of the following exception:
  File "/home/ephios/venv/lib/python3.10/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/core/handlers/base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 104, in view
    return self.dispatch(request, *args, **kwargs)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/contrib/auth/mixins.py", line 109, in dispatch
    return super().dispatch(request, *args, **kwargs)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/views/generic/base.py", line 143, in dispatch
    return handler(request, *args, **kwargs)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/views/event.py", line 329, in post
    return self.form_valid(form)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/extra/mixins.py", line 145, in form_valid
    response = super().form_valid(form)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/views/generic/edit.py", line 133, in form_valid
    self.object = form.save()
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/forms/events.py", line 115, in save
    event: Event = super().save(commit=commit)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/forms/models.py", line 552, in save
    self.instance.save()
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/base.py", line 822, in save
    self.save_base(
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/base.py", line 909, in save_base
    updated = self._save_table(
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1036, in _save_table
    updated = self._do_update(
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/base.py", line 1101, in _do_update
    return filtered._update(values) > 0
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/query.py", line 1278, in _update
    return query.get_compiler(self.db).execute_sql(CURSOR)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1990, in execute_sql
    cursor = super().execute_sql(result_type)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/models/sql/compiler.py", line 1562, in execute_sql
    cursor.execute(sql, params)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 79, in execute
    return self._execute_with_wrappers(
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 92, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 100, in _execute
    with self.db.wrap_database_errors:
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/utils.py", line 91, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/db/backends/mysql/base.py", line 76, in execute
    return self.cursor.execute(query, args)
  File "/home/ephios/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 179, in execute
    res = self._query(mogrified_query)
  File "/home/ephios/venv/lib/python3.10/site-packages/MySQLdb/cursors.py", line 330, in _query
    db.query(q)
  File "/home/ephios/venv/lib/python3.10/site-packages/MySQLdb/connections.py", line 255, in query
    _mysql.connection.query(self, query)

Exception Type: OperationalError at /events/540/edit/
Exception Value: (1366, "Incorrect string value: '\\xF0\\x9F\\xAA\\xA2\\x0D\\x0A...' for column `ephiosdb`.`event`.`description` at row 1")
Raised during: ephios.core.views.event.EventUpdateView
Request information:
USER: *******

GET: No GET data

POST:
csrfmiddlewaretoken = '*******'
title = '*******'
type = '4'
description = '******* 🪢\r\n🏕️ ******* ☺️'
location = *******'
visible_for = '2'
responsible_groups = '5'
guests-active = 'on'

FILES: No FILES data

COOKIES:
csrftoken = '********************'
sessionid = '********************'
django_language = 'de'

META:
CONTENT_LENGTH = '1078'
CONTENT_TYPE = 'application/x-www-form-urlencoded'
CSRF_COOKIE = 'XtUx2QwIQqN3Hh7CQiEY8gzssMvB1DUx'
HTTP_ACCEPT = 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7'
HTTP_ACCEPT_ENCODING = 'gzip, deflate, br'
HTTP_ACCEPT_LANGUAGE = 'de-DE,de;q=0.9,en-US;q=0.8,en;q=0.7'
HTTP_CACHE_CONTROL = 'max-age=0'
HTTP_CONNECTION = 'Keep-Alive'
HTTP_COOKIE = '********************'
HTTP_HOST = '*******'
HTTP_ORIGIN = '*******'
HTTP_REFERER = '*******/events/540/edit/'
HTTP_SEC_CH_UA = '"Not A(Brand";v="99", "Google Chrome";v="121", "Chromium";v="121"'
HTTP_SEC_CH_UA_MOBILE = '?1'
HTTP_SEC_CH_UA_PLATFORM = '*******"'
HTTP_SEC_FETCH_DEST = 'empty'
HTTP_SEC_FETCH_MODE = 'navigate'
HTTP_SEC_FETCH_SITE = 'same-origin'
HTTP_UPGRADE_INSECURE_REQUESTS = '1'
HTTP_USER_AGENT = '*******'
HTTP_X_FORWARDED_FOR = '*******'
HTTP_X_FORWARDED_HOST = '*******'
HTTP_X_FORWARDED_PROTO = 'https'
HTTP_X_FORWARDED_SERVER = '*******'
PATH_INFO = '/events/540/edit/'
QUERY_STRING = ''
RAW_URI = '/events/540/edit/'
REMOTE_ADDR = '*******'
REMOTE_PORT = '*******'
REQUEST_METHOD = 'POST'
SCRIPT_NAME = ''
SERVER_NAME = '*******'
SERVER_PORT = '*******'
SERVER_PROTOCOL = 'HTTP/1.1'
SERVER_SOFTWARE = 'gunicorn/21.2.0'
gunicorn.socket = <socket.socket fd=10, family=AddressFamily.AF_INET, type=SocketKind.SOCK_STREAM, proto=0, laddr=('127.0.0.1', 8327), raddr=('127.0.0.1', 42690)>
wsgi.errors = <gunicorn.http.wsgi.WSGIErrorsWrapper object at 0x7f81bf8f9270>
wsgi.file_wrapper = <class 'gunicorn.http.wsgi.FileWrapper'>
wsgi.input = <gunicorn.http.body.Body object at 0x7f81bf938c70>
wsgi.input_terminated = True
wsgi.multiprocess = True
wsgi.multithread = False
wsgi.run_once = False
wsgi.url_scheme = 'https'
wsgi.version = '(1, 0)'

Settings:
Using settings module ephios.settings
ABSOLUTE_URL_OVERRIDES = {}
ADMINS = [('', '*******')]
ALLOWED_HOSTS = ['localhost', '*******']
ANONYMOUS_USER_NAME = None
APPEND_SLASH = True
AUTHENTICATION_BACKENDS = ['django.contrib.auth.backends.ModelBackend', 'guardian.backends.ObjectPermissionBackend', 'ephios.extra.auth.EphiosOIDCAB']
AUTH_PASSWORD_VALIDATORS = '********************'
AUTH_USER_MODEL = 'core.UserProfile'
BASE_DIR = '/home/ephios/venv/lib/python3.10/site-packages'
CACHES = {'default': {'TIMEOUT': 43200, 'BACKEND': 'django.core.cache.backends.redis.RedisCache', 'LOCATION': 'redis://127.0.0.1:6379/7'}}
CACHE_MIDDLEWARE_ALIAS = 'default'
CACHE_MIDDLEWARE_KEY_PREFIX = '********************'
CACHE_MIDDLEWARE_SECONDS = 600
COMPRESSORS = {'css': 'compressor.css.CssCompressor', 'js': 'compressor.js.JsCompressor'}
COMPRESS_CACHEABLE_PRECOMPILERS = '()'
COMPRESS_CACHE_BACKEND = 'default'
COMPRESS_CACHE_KEY_FUNCTION = '********************'
COMPRESS_CLEAN_CSS_ARGUMENTS = ''
COMPRESS_CLEAN_CSS_BINARY = 'cleancss'
COMPRESS_CLOSURE_COMPILER_ARGUMENTS = ''
COMPRESS_CLOSURE_COMPILER_BINARY = 'java -jar compiler.jar'
COMPRESS_CSS_HASHING_METHOD = 'mtime'
COMPRESS_DATA_URI_MAX_SIZE = 1024
COMPRESS_DEBUG_TOGGLE = None
COMPRESS_ENABLED = True
COMPRESS_FILTERS = {'css': ['compressor.filters.css_default.CssAbsoluteFilter', 'compressor.filters.cssmin.rCSSMinFilter'], 'js': ['compressor.filters.jsmin.rJSMinFilter']}
COMPRESS_JINJA2_GET_ENVIRONMENT = <function CompressorConf.JINJA2_GET_ENVIRONMENT at 0x7f81c6f4c700>
COMPRESS_MINT_DELAY = 30
COMPRESS_MTIME_DELAY = 10
COMPRESS_OFFLINE = False
COMPRESS_OFFLINE_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_OFFLINE_MANIFEST = 'manifest.json'
COMPRESS_OFFLINE_MANIFEST_STORAGE = 'compressor.storage.OfflineManifestFileStorage'
COMPRESS_OFFLINE_TIMEOUT = 31536000
COMPRESS_OUTPUT_DIR = 'CACHE'
COMPRESS_PARSER = 'compressor.parser.AutoSelectParser'
COMPRESS_PRECOMPILERS = "(('text/x-scss', 'django_libsass.SassCompiler'),)"
COMPRESS_REBUILD_TIMEOUT = 2592000
COMPRESS_ROOT = '/var/ephios/data/public/static'
COMPRESS_STORAGE = 'compressor.storage.CompressorFileStorage'
COMPRESS_TEMPLATE_FILTER_CONTEXT = {'STATIC_URL': '/static/'}
COMPRESS_URL = '/static/'
COMPRESS_URL_PLACEHOLDER = '/__compressor_url_placeholder__/'
COMPRESS_VERBOSE = False
COMPRESS_YUGLIFY_BINARY = 'yuglify'
COMPRESS_YUGLIFY_CSS_ARGUMENTS = '--terminal'
COMPRESS_YUGLIFY_JS_ARGUMENTS = '--terminal'
COMPRESS_YUI_BINARY = 'java -jar yuicompressor.jar'
COMPRESS_YUI_CSS_ARGUMENTS = ''
COMPRESS_YUI_JS_ARGUMENTS = ''
CONN_MAX_AGE = 0
CORE_PLUGINS = ['ephios.plugins.basesignup.apps.PluginApp', 'ephios.plugins.complexsignup.apps.PluginApp', 'ephios.plugins.pages.apps.PluginApp', 'ephios.plugins.qualification_management.apps.PluginApp', 'ephios.plugins.guests.apps.PluginApp', 'ephios.plugins.eventautoqualification.apps.PluginApp', 'ephios.plugins.simpleresource.apps.PluginApp', 'ephios.plugins.federation.apps.PluginApp']
CRISPY_ALLOWED_TEMPLATE_PACKS = "('bootstrap5',)"
CRISPY_FAIL_SILENTLY = True
CRISPY_TEMPLATE_PACK = 'bootstrap5'
CSP_IMG_SRC = '("\'self\'", \'data:\')'
CSP_INCLUDE_NONCE_IN = ['style-src']
CSP_STYLE_SRC = '("\'self\'",)'
CSRF_COOKIE_AGE = 31449600
CSRF_COOKIE_DOMAIN = None
CSRF_COOKIE_HTTPONLY = False
CSRF_COOKIE_NAME = 'csrftoken'
CSRF_COOKIE_PATH = '/'
CSRF_COOKIE_SAMESITE = 'Lax'
CSRF_COOKIE_SECURE = True
CSRF_FAILURE_VIEW = 'django.views.csrf.csrf_failure'
CSRF_HEADER_NAME = 'HTTP_X_CSRFTOKEN'
CSRF_TRUSTED_ORIGINS = []
CSRF_USE_SESSIONS = False
DATABASES = {'default': {'NAME': '*******', 'USER': '*******', 'PASSWORD': '********************', 'HOST': 'localhost', 'PORT': *******, 'ENGINE': 'django.db.backends.mysql', 'ATOMIC_REQUESTS': False, 'AUTOCOMMIT': True, 'CONN_MAX_AGE': 0, 'CONN_HEALTH_CHECKS': False, 'OPTIONS': {}, 'TIME_ZONE': None, 'TEST': {'CHARSET': None, 'COLLATION': None, 'MIGRATE': True, 'MIRROR': None, 'NAME': None}}}
DATABASE_ROUTERS = []
DATA_DIR = '/var/ephios/data'
DATA_UPLOAD_MAX_MEMORY_SIZE = 2621440
DATA_UPLOAD_MAX_NUMBER_FIELDS = 1000
DATA_UPLOAD_MAX_NUMBER_FILES = 100
DATETIME_FORMAT = 'N j, Y, P'
DATETIME_INPUT_FORMATS = ['%Y-%m-%d %H:%M:%S', '%Y-%m-%d %H:%M:%S.%f', '%Y-%m-%d %H:%M', '%m/%d/%Y %H:%M:%S', '%m/%d/%Y %H:%M:%S.%f', '%m/%d/%Y %H:%M', '%m/%d/%y %H:%M:%S', '%m/%d/%y %H:%M:%S.%f', '%m/%d/%y %H:%M']
DATE_FORMAT = 'N j, Y'
DATE_INPUT_FORMATS = ['%Y-%m-%d', '%m/%d/%Y', '%m/%d/%y', '%b %d %Y', '%b %d, %Y', '%d %b %Y', '%d %b, %Y', '%B %d %Y', '%B %d, %Y', '%d %B %Y', '%d %B, %Y']
DEBUG = False
DEBUG_PROPAGATE_EXCEPTIONS = False
DECIMAL_SEPARATOR = '.'
DEFAULT_AUTO_FIELD = 'django.db.models.BigAutoField'
DEFAULT_CHARSET = 'utf-8'
DEFAULT_EXCEPTION_REPORTER = 'django.views.debug.ExceptionReporter'
DEFAULT_EXCEPTION_REPORTER_FILTER = 'django.views.debug.SafeExceptionReporterFilter'
DEFAULT_FILE_STORAGE = 'django.core.files.storage.FileSystemStorage'
DEFAULT_FROM_EMAIL = '*******'
DEFAULT_INDEX_TABLESPACE = ''
DEFAULT_LISTVIEW_PAGINATION = 100
DEFAULT_TABLESPACE = ''
DIRECTORIES = {'DATA_DIR': '/var/ephios/data', 'PUBLIC_DIR': '/var/ephios/data/public', 'STATIC_ROOT': '/var/ephios/data/public/static', 'PRIVATE_DIR': '/var/ephios/data/private', 'LOG_DIR': '/var/ephios/data/private/logs', 'MEDIA_ROOT': '/var/ephios/data/private/media'}
DISALLOWED_USER_AGENTS = []
DJANGO_REDIS_IGNORE_EXCEPTIONS = True
DJANGO_REDIS_LOG_IGNORED_EXCEPTIONS = True
EMAIL_BACKEND = 'django.core.mail.backends.smtp.EmailBackend'
EMAIL_CONFIG = {'EMAIL_FILE_PATH': '', 'EMAIL_HOST_USER': '*******', 'EMAIL_HOST_PASSWORD': '*******', 'EMAIL_HOST': 'mail.your-server.de', 'EMAIL_PORT': 587, 'EMAIL_BACKEND': 'django.core.mail.backends.smtp.EmailBackend', 'EMAIL_USE_TLS': True}
EMAIL_FILE_PATH = ''
EMAIL_HOST = '*******'
EMAIL_HOST_PASSWORD = '*******'
EMAIL_HOST_USER = '*******'
EMAIL_PORT = *******
EMAIL_SSL_CERTFILE = None
EMAIL_SSL_KEYFILE = '*******'
EMAIL_SUBJECT_PREFIX = '[Django] '
EMAIL_TIMEOUT = None
EMAIL_USE_LOCALTIME = False
EMAIL_USE_SSL = False
EMAIL_USE_TLS = True
ENABLE_DEBUG_TOOLBAR = False
EPHIOS_APP_MODULES = ['ephios.core', 'ephios.extra', 'ephios.api']
EPHIOS_VERSION = '0.16.0'
FILE_UPLOAD_DIRECTORY_PERMISSIONS = None
FILE_UPLOAD_HANDLERS = ['django.core.files.uploadhandler.MemoryFileUploadHandler', 'django.core.files.uploadhandler.TemporaryFileUploadHandler']
FILE_UPLOAD_MAX_MEMORY_SIZE = 2621440
FILE_UPLOAD_PERMISSIONS = 420
FILE_UPLOAD_TEMP_DIR = None
FIRST_DAY_OF_WEEK = 0
FIXTURE_DIRS = []
FORCE_SCRIPT_NAME = None
FORMAT_MODULE_PATH = None
FORMS_URLFIELD_ASSUME_HTTPS = False
FORM_RENDERER = 'django.forms.renderers.DjangoTemplates'
GET_SITE_URL = <function GET_SITE_URL at 0x7f81c83a8280>
GUARDIAN_MONKEY_PATCH = '*******'
IGNORABLE_404_URLS = []
INSTALLED_APPS = ['ephios.core', 'ephios.extra', 'ephios.api', 'django.contrib.admin', 'django.contrib.auth', 'polymorphic', 'django.contrib.contenttypes', 'django.contrib.sessions', 'django.contrib.messages', 'django.contrib.staticfiles', 'django.contrib.humanize', 'django_filters', 'guardian', 'oauth2_provider', 'rest_framework', 'django_select2', 'djangoformsetjs', 'compressor', 'recurrence', 'statici18n', 'dynamic_preferences.users.apps.UserPreferencesConfig', 'crispy_forms', 'crispy_bootstrap5', 'webpush', 'ephios.modellogging', 'ephios.plugins.basesignup.apps.PluginApp', 'ephios.plugins.complexsignup.apps.PluginApp', 'ephios.plugins.pages.apps.PluginApp', 'ephios.plugins.qualification_management.apps.PluginApp', 'ephios.plugins.guests.apps.PluginApp', 'ephios.plugins.eventautoqualification.apps.PluginApp', 'ephios.plugins.simpleresource.apps.PluginApp', 'ephios.plugins.federation.apps.PluginApp', 'dynamic_preferences']
INTERNAL_IPS = []
LANGUAGES = [('de', 'Deutsch'), ('en', 'Englisch')]
LANGUAGES_BIDI = ['he', 'ar', 'ar-dz', 'ckb', 'fa', 'ug', 'ur']
LANGUAGE_CODE = 'de'
LANGUAGE_COOKIE_AGE = None
LANGUAGE_COOKIE_DOMAIN = None
LANGUAGE_COOKIE_HTTPONLY = False
LANGUAGE_COOKIE_NAME = 'django_language'
LANGUAGE_COOKIE_PATH = '/'
LANGUAGE_COOKIE_SAMESITE = None
LANGUAGE_COOKIE_SECURE = False
LOCALE_PATHS = ['/home/ephios/venv/lib/python3.10/site-packages/ephios/locale']
LOGGING = {'version': 1, 'disable_existing_loggers': False, 'formatters': {'default': {'format': '[%(levelname)s] %(asctime)s %(name)s :: %(message)s'}}, 'filters': {'require_debug_false': {'()': 'django.utils.log.RequireDebugFalse'}, 'require_debug_true': {'()': 'django.utils.log.RequireDebugTrue'}}, 'handlers': {'console': {'level': 'DEBUG', 'class': 'logging.StreamHandler', 'formatter': 'default', 'filters': ['require_debug_true']}, 'mail_admins': {'level': 'ERROR', 'filters': ['require_debug_false'], 'class': 'django.utils.log.AdminEmailHandler'}, 'file': {'level': 'DEBUG', 'formatter': 'default', 'filters': [], 'class': 'logging.handlers.TimedRotatingFileHandler', 'filename': '/var/ephios/data/private/logs/ephios.log', 'when': 'midnight', 'backupCount': 14, 'atTime': datetime.time(4, 0), 'encoding': 'utf-8'}}, 'loggers': {'ephios': {'handlers': ['mail_admins', 'console', 'file'], 'level': 'INFO', 'propagate': False}, 'django': {'handlers': [], 'level': 'INFO', 'propagate': True}, 'django.server': {'handlers': [], 'level': 'INFO', 'propagate': True}}, 'root': {'handlers': ['mail_admins', 'console', 'file'], 'level': 'INFO'}}
LOGGING_CONFIG = 'logging.config.dictConfig'
LOGIN_REDIRECT_URL = '/'
LOGIN_URL = '/accounts/login/'
LOGOUT_REDIRECT_URL = None
LOG_DIR = '/var/ephios/data/private/logs'
MANAGERS = []
MEDIA_ROOT = '/var/ephios/data/private/media'
MEDIA_URL = '/'
MESSAGE_STORAGE = 'django.contrib.messages.storage.fallback.FallbackStorage'
MESSAGE_TAGS = {10: 'alert-info', 20: 'alert-info', 25: 'alert-success', 30: 'alert-warning', 40: 'alert-danger'}
MIDDLEWARE = ['django.middleware.security.SecurityMiddleware', 'django.contrib.sessions.middleware.SessionMiddleware', 'django.middleware.locale.LocaleMiddleware', 'ephios.extra.middleware.EphiosLocaleMiddleware', 'ephios.extra.middleware.EphiosNotificationMiddleware', 'django.middleware.common.CommonMiddleware', 'django.middleware.csrf.CsrfViewMiddleware', 'django.contrib.auth.middleware.AuthenticationMiddleware', 'django.contrib.messages.middleware.MessageMiddleware', 'django.middleware.clickjacking.XFrameOptionsMiddleware', 'csp.middleware.CSPMiddleware', 'ephios.modellogging.middleware.LoggingRequestMiddleware']
MIGRATION_MODULES = {}
MONTH_DAY_FORMAT = 'F j'
NUMBER_GROUPING = 0
OAUTH2_PROVIDER = {'SCOPES': {'PUBLIC_READ': 'Öffentliche Daten (bspw. Veranstaltungen und Schichten) lesen', 'PUBLIC_WRITE': 'Öffentliche Daten (bspw. Veranstaltungen und Schichten) schreiben', 'ME_READ': 'Eigene Daten und Teilnahmen lesen', 'ME_WRITE': 'Eigene Daten und Teilnahmen schreiben', 'CONFIDENTIAL_READ': 'Vertrauliche Daten (bpsw. fremde Nutzerprofile und Teilnahmen) lesen', 'CONFIDENTIAL_WRITE': 'Vertrauliche Daten (bpsw. fremde Nutzerprofile und Teilnahmen) schreiben'}, 'REFRESH_TOKEN_EXPIRE_SECONDS': '*******'}
OAUTH2_PROVIDER_ACCESS_TOKEN_MODEL = '*******'
OAUTH2_PROVIDER_APPLICATION_MODEL = 'api.Application'
OAUTH2_PROVIDER_GRANT_MODEL = 'api.Grant'
OAUTH2_PROVIDER_ID_TOKEN_MODEL = '*******'
OAUTH2_PROVIDER_REFRESH_TOKEN_MODEL = '*******'
PASSWORD_HASHERS = '*******'
PASSWORD_RESET_TIMEOUT = '*******'
PLUGINS = ['ephios.plugins.basesignup.apps.PluginApp', 'ephios.plugins.complexsignup.apps.PluginApp', 'ephios.plugins.pages.apps.PluginApp', 'ephios.plugins.qualification_management.apps.PluginApp', 'ephios.plugins.guests.apps.PluginApp', 'ephios.plugins.eventautoqualification.apps.PluginApp', 'ephios.plugins.simpleresource.apps.PluginApp', 'ephios.plugins.federation.apps.PluginApp']
PREPEND_WWW = False
PRIVATE_DIR = '/var/ephios/data/private'
PUBLIC_DIR = '/var/ephios/data/public'
PWA_APP_ICONS = [{'src': '/static/ephios/img/ephios-192x.png', 'sizes': '192x192', 'type': 'image/png', 'purpose': 'maskable'}, {'src': '/static/ephios/img/ephios-512x.png', 'sizes': '512x512', 'type': 'image/png', 'purpose': 'maskable'}, {'src': '/static/ephios/img/ephios-1024x.png', 'sizes': '1024x1024', 'type': 'image/png', 'purpose': 'maskable'}, {'src': '/static/ephios/img/ephios-symbol-red.svg', 'sizes': 'any', 'type': 'image/svg+xml', 'purpose': 'any'}]
REST_FRAMEWORK = {'DEFAULT_PERMISSION_CLASSES': ['rest_framework.permissions.DjangoObjectPermissions'], 'DEFAULT_VERSIONING_CLASS': 'rest_framework.versioning.NamespaceVersioning', 'DEFAULT_PAGINATION_CLASS': 'rest_framework.pagination.LimitOffsetPagination', 'PAGE_SIZE': 100, 'DEFAULT_AUTHENTICATION_CLASSES': ['rest_framework.authentication.SessionAuthentication', 'ephios.api.access.auth.CustomOAuth2Authentication']}
ROOT_URLCONF = 'ephios.urls'
RUN_PERIODIC_MAX_INTERVAL = 330
*******
SELECT2_CACHE_BACKEND = 'default'
SELECT2_CACHE_PREFIX = 'select2_'
SELECT2_CSS = ''
SELECT2_I18N_AVAILABLE_LANGUAGES = ['ar', 'az', 'bg', 'ca', 'cs', 'da', 'de', 'el', 'en', 'es', 'et', 'eu', 'fa', 'fi', 'fr', 'gl', 'he', 'hi', 'hr', 'hu', 'id', 'is', 'it', 'ja', 'km', 'ko', 'lt', 'lv', 'mk', 'ms', 'nb', 'nl', 'pl', 'pt-BR', 'pt', 'ro', 'ru', 'sk', 'sr-Cyrl', 'sr', 'sv', 'th', 'tr', 'uk', 'vi', 'zh-CN', 'zh-TW']
SELECT2_I18N_PATH = ''
SELECT2_JS = ''
SELECT2_JSON_ENCODER = 'django.core.serializers.json.DjangoJSONEncoder'
SELECT2_THEME = 'bootstrap-5'
SERVER_EMAIL = '*******m'
SESSION_CACHE_ALIAS = 'default'
SESSION_COOKIE_AGE = 1209600
SESSION_COOKIE_DOMAIN = None
SESSION_COOKIE_HTTPONLY = True
SESSION_COOKIE_NAME = 'sessionid'
SESSION_COOKIE_PATH = '/'
SESSION_COOKIE_SAMESITE = 'Lax'
SESSION_COOKIE_SECURE = True
SESSION_ENGINE = 'django.contrib.sessions.backends.cached_db'
SESSION_EXPIRE_AT_BROWSER_CLOSE = False
SESSION_FILE_PATH = None
SESSION_SAVE_EVERY_REQUEST = False
SESSION_SERIALIZER = 'django.contrib.sessions.serializers.JSONSerializer'
SETTINGS_MODULE = 'ephios.settings'
SHORT_DATETIME_FORMAT = 'm/d/Y P'
SHORT_DATE_FORMAT = 'm/d/Y'
SIGNING_BACKEND = 'django.core.signing.TimestampSigner'
SILENCED_SYSTEM_CHECKS = []
STATICFILES_DIRS = "('/home/ephios/venv/lib/python3.10/site-packages/ephios/static',)"
STATICFILES_FINDERS = "('django.contrib.staticfiles.finders.FileSystemFinder', 'django.contrib.staticfiles.finders.AppDirectoriesFinder', 'compressor.finders.CompressorFinder')"
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.StaticFilesStorage'
STATICI18N_DOMAIN = 'djangojs'
STATICI18N_FILENAME_FUNCTION = 'statici18n.utils.default_filename'
STATICI18N_NAMESPACE = None
STATICI18N_OUTPUT_DIR = 'jsi18n'
STATICI18N_PACKAGES = 'django.conf'
STATICI18N_ROOT = '/var/ephios/data/public/static'
STATIC_ROOT = '/var/ephios/data/public/static'
STATIC_URL = '/static/'
STORAGES = {'default': {'BACKEND': 'django.core.files.storage.FileSystemStorage'}, 'staticfiles': {'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage'}}
TEMPLATES = [{'BACKEND': 'django.template.backends.django.DjangoTemplates', 'DIRS': ['/home/ephios/venv/lib/python3.10/site-packages/ephios/templates'], 'APP_DIRS': True, 'OPTIONS': {'context_processors': ['django.template.context_processors.debug', 'django.template.context_processors.request', 'django.contrib.auth.context_processors.auth', 'django.contrib.messages.context_processors.messages', 'django.template.context_processors.i18n', 'dynamic_preferences.processors.global_preferences', 'ephios.core.context.ephios_base_context'], 'debug': False}}]
TEST_NON_SERIALIZED_APPS = []
TEST_RUNNER = 'django.test.runner.DiscoverRunner'
THOUSAND_SEPARATOR = ','
TIME_FORMAT = 'P'
TIME_INPUT_FORMATS = ['%H:%M:%S', '%H:%M:%S.%f', '%H:%M']
TIME_ZONE = 'Europe/Berlin'
USE_I18N = True
USE_THOUSAND_SEPARATOR = False
USE_TZ = True
USE_X_FORWARDED_HOST = False
USE_X_FORWARDED_PORT = False
VAPID_PRIVATE_KEY_PATH = '*******'
WEBPUSH_SETTINGS = {'VAPID_PUBLIC_KEY': '*******', 'VAPID_PRIVATE_KEY': '*******', 'VAPID_ADMIN_EMAIL': '*******'}
WSGI_APPLICATION = 'ephios.wsgi.application'
X_FRAME_OPTIONS = 'DENY'
YEAR_MONTH_FORMAT = 'F Y'

Secound error message "[Django] Notification sending failed":

Exception: 
Traceback (most recent call last):
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/services/notifications/backends.py", line 52, in send_all_notifications
    backend.send_multiple(unprocessed_notifications)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/services/notifications/backends.py", line 121, in send_multiple
    cls.send(notification)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/services/notifications/backends.py", line 153, in send
    plaintext=notification.as_plaintext(),
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/models/users.py", line 547, in as_plaintext
    return self.notification_type.as_plaintext(self)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/services/notifications/types.py", line 68, in as_plaintext
    return render_to_string(cls.plaintext_template_name, cls.get_render_context(notification))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/loader.py", line 62, in render_to_string
    return template.render(context, request)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 159, in render
    return compiled_parent._render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/loader_tags.py", line 65, in render
    result = block.nodelist.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/defaulttags.py", line 193, in render
    values = self.sequence.resolve(context, ignore_failures=True)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 710, in resolve
    obj = self.var.resolve(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 842, in resolve
    value = self._resolve_lookup(context)
  File "/home/ephios/venv/lib/python3.10/site-packages/django/template/base.py", line 909, in _resolve_lookup
    current = current()
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/models/users.py", line 550, in get_actions
    return self.notification_type.get_actions_with_referrer(self)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/services/notifications/types.py", line 99, in get_actions_with_referrer
    for label, url in cls.get_actions(notification):
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/services/notifications/types.py", line 239, in get_actions
    participation.participant.reverse_event_detail(participation.shift.event)
  File "/home/ephios/venv/lib/python3.10/site-packages/ephios/core/signup/participants.py", line 109, in reverse_event_detail
    raise NotImplementedError
NotImplementedError
felixrindt commented 8 months ago

First error message "[Django] ERROR (EXTERNAL IP): Internal Server Error: /events/540/edit/":

description = ' 🪢\r\n🏕️ ☺️'

This does seem to be a problem with your database encoding setting. Try looking into the operational (database) error to figure out how to fix the database configuration: OperationalError at /events/540/edit/ (1366, "Incorrect string value: '\\xF0\\x9F\\xAA\\xA2\\x0D\\x0A...' for columnephiosdb.event.descriptionat row 1"). We don't have any problems storing emoji etc. on our end.

Secound error message "[Django] Notification sending failed":

To me, this seems to be an unrelated error, though something I haven't seen before. Haven't seen it before. Would need to investigate the concrete Notification that is causing this issue.

pov91 commented 8 months ago

The database has the character set utf8mb4 and the collate utf8mb4_unicode_ci everywhere. What else could be set incorrectly? How can I display more information about the error and what is set incorrectly?

The error messages with "[Django] Notification sending failed" started exactly at the time the other error occurred.

Thanks for the help!

felixrindt commented 7 months ago

I'm closing this as this is not an issue in ephios itself. I hope you can get your database fixed, though. We're not that experienced with mariadb/mysql though :/