Integration in Django throws many KeyError 'request' on each issue report

NicoEdge5 commented 2 months ago

How do you use Sentry?

Sentry Saas (sentry.io)

Version

2.13.0

Steps to Reproduce

I am going nuts with this issue which I believe is coming from sentry-sdk for python, probably in combination with some other dependencies. I have a project in Django 4.2 with the sentry-sdk 2.13.0 which throws a KeyError request each time something is triggered on sentry, meaning if an error is reported I get X more errors reported about KeyError request (yes it is not just 1 for each report done). What confuses me is that the original error is shown correctly on sentry so I cannot say that sentry-sdk is failing reporting the issue. Moreover this happens also whenever I manually trigger a info/warning report from code to sentry.

To note, this started happening after upgrading the project dependencies, especially Django to 4.2. I am also not sure it is a bug directly on sentry_sdk probably is the combination with something else.

I also thought it could have been a context_processor issue but context_processor for request is in the settings (see code below)

Any help or suggestion is much appreciated.

import sentry_sdk
from sentry_sdk.integrations.django import DjangoIntegration

sentry_sdk.init(
    dsn="XXXX",
    integrations=[DjangoIntegration()],
    server_name='XXXX',

    send_default_pii=True
)

Stack trace:

KeyError
'request'

django/template/context.py in __getitem__ at line 83

cms/templatetags/cms_tags.py in _get_empty_context at line 636

cms/templatetags/cms_tags.py in get_context at line 829

cms/templatetags/cms_tags.py in render_tag at line 810

classytags/core.py in render at line 142

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

django/template/loader_tags.py in render at line 54

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

django/template/base.py in _render at line 167

django/template/base.py in render at line 177

django/template/loader_tags.py in render at line 208

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

django/template/loader_tags.py in render at line 63

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

cms/templatetags/cms_tags.py in render_tag at line 426

classytags/core.py in render at line 142

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

sekizai/templatetags/sekizai_tags.py in render_tag at line 86

classytags/core.py in render at line 142

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

django/template/base.py in _render at line 167

django/template/loader_tags.py in render at line 157

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

django/template/base.py in _render at line 167

django/template/loader_tags.py in render at line 157

django/template/base.py in render_annotated at line 966

django/template/base.py in render at line 1005

django/template/base.py in _render at line 167

django/template/base.py in render at line 175

django/template/backends/django.py in render at line 61

django/views/defaults.py in server_error at line 99

django/utils/decorators.py in _wrapper_view at line 134

django/core/handlers/exception.py in handle_uncaught_exception at line 185

django/core/handlers/exception.py in response_for_exception at line 140

django/core/handlers/exception.py in inner at line 57

django/utils/deprecation.py in __call__ at line 134

django/core/handlers/exception.py in inner at line 55

Context processors:

TEMPLATES = [
    {
        'BACKEND': 'django.template.backends.django.DjangoTemplates',
        'DIRS': [
            ...
        ],
        'OPTIONS': {
            'context_processors': [
                'django.contrib.auth.context_processors.auth',
                'django.template.context_processors.debug',
                'django.template.context_processors.i18n',
                'django.template.context_processors.media',
                'django.template.context_processors.static',
                'django.template.context_processors.tz',
                'django.contrib.messages.context_processors.messages',
                'django.template.context_processors.request',
                'sekizai.context_processors.sekizai',
                'cms.context_processors.cms_settings',
                'website.context_processors.google_analytics'
            ],
            'loaders': [
                'django.template.loaders.filesystem.Loader',
                'django.template.loaders.app_directories.Loader',
                'admin_tools.template_loaders.Loader',
            ],
        },
    }
]

pip freeze --all:

amqp==5.2.0
asgiref==3.8.1
attrs==24.2.0
billiard==4.2.0
Brotli==1.1.0
cachetools==5.5.0
celery==5.4.0
certifi==2024.7.4
cffi==1.17.0
chardet==5.2.0
charset-normalizer==3.3.2
click==8.1.7
click-didyoumean==0.3.1
click-plugins==1.1.1
click-repl==0.3.0
cron-descriptor==1.4.3
cssselect==1.2.0
cssselect2==0.7.0
cssutils==2.11.1
decorator==5.1.1
dj-database-url==2.2.0
Django==4.2.10
django-admin-tools==0.9.3
django-appconf==1.0.6
django-celery-beat==2.6.0
django-ckeditor==6.7.1
django-classy-tags==4.1.0
django-cms==3.11.6
django-countries==7.6.1
django-easy-select2==1.5.8
django-export-csv==0.1.2
django-filer==3.1.4
django-formtools==2.5.1
django-image-cropping==1.7
django-imagekit==5.0.0
django-js-asset==2.2.0
django-materializecss-form==1.1.17
django-modeltranslation==0.19.7
django-otp==1.5.2
django-phonenumber-field==8.0.0
django-polymorphic==3.1.0
django-ranged-response==0.2.0
django-sekizai==4.1.0
django-simple-captcha==0.6.0
django-simple-mail==2.3.5
django-timezone-field==7.0
django-treebeard==4.7.1
django-two-factor-auth==1.17.0
django-weasyprint==2.3.0
djangocms-admin-style==3.3.1
djangocms-attributes-field==3.0.0
djangocms-column==2.1.0
djangocms-file==3.0.1
djangocms-googlemap==2.2.0
djangocms-link==4.0.0
djangocms-picture==4.1.1
djangocms-snippet==3.1.1
djangocms-style==3.1.0
djangocms-text-ckeditor==5.1.5
djangocms-video==3.1.0
djangorestframework==3.15.2
easy-thumbnails==2.9
et-xmlfile==1.1.0
fonttools==4.53.1
future==1.0.0
geocoder==1.38.1
gunicorn==23.0.0
html2text==2024.2.26
html5lib==1.1
idna==3.7
isodate==0.6.1
kombu==5.4.0
lxml==5.3.0
more-itertools==10.4.0
openpyxl==3.1.5
packaging==24.1
phonenumbers==8.13.43
pilkit==3.0
pillow==10.4.0
pip==24.0
platformdirs==4.2.2
premailer==3.10.0
prompt_toolkit==3.0.47
psycopg2==2.9.9
pycparser==2.22
pydyf==0.11.0
pyphen==0.16.0
pypng==0.20220715.0
python-crontab==3.2.0
python-dateutil==2.9.0.post0
pytz==2024.1
qrcode==7.4.2
ratelim==0.1.6
redis==5.0.8
reportlab==4.2.2
requests==2.32.3
requests-file==2.1.0
requests-toolbelt==1.0.0
sentry-sdk==2.13.0
setuptools==69.1.1
six==1.16.0
sorl-thumbnail==12.10.0
sqlparse==0.5.1
svglib==1.5.1
tinycss2==1.3.0
typing_extensions==4.12.2
tzdata==2024.1
unicodecsv==0.14.1
urllib3==2.2.2
vine==5.1.0
wcwidth==0.2.13
weasyprint==62.3
webencodings==0.5.1
wheel==0.43.0
XlsxWriter==3.2.0
zeep==4.2.1
zopfli==0.2.3

Expected Result

Report the issue on sentry.

Actual Result

Reports the error followed by many KeyError 'request' triggered by the first error.

sl0thentr0py commented 2 months ago

@NicoEdge5, could you maybe send us links to the sentry errors for both:

the original exception you expect to be seen
one of the others that is triggerred by that

Seeing both of the stacktraces will help us debug better. If you don't want to send the error links publicly here on github, you can email them to me at neel.shah at sentry.io.

NicoEdge5 commented 2 months ago

@sl0thentr0py thanks for your feedback. I sent the requested data to you. I will update the issue here as soon as there is any progress for other people's sake (in case they have same or similar issue).

sl0thentr0py commented 2 months ago

yes I received the email, thank you! I will set aside some time next week to debug your issue.

lbillon commented 2 weeks ago

Hello! I have been facing the very same issue since August, before and after upgrading Django to the latest versions.

Did you get a chance to make progress on that issue ?

Are all Django customers affected or is this an issue on the customer side?

Kind regards

NicoEdge5 commented 2 weeks ago

Hi @lbillon,

unfortunately no progress at the moment. I am still waiting also input from @sl0thentr0py As soon as I have an update I will share with everyone.

lbillon commented 2 weeks ago

I set out to investigate and reproduce the issue, and after some digging, I suspect it's not directly related to sentry-sdk itself.

Here’s what I found:

The KeyError messages also appeared in the logs independently of Sentry, so it seems not directly linked to sentry-sdk.
The issue only surfaces when DEBUG = False, which is typical of production environments where Sentry is active. With DEBUG = True, when a 500 error occurs, Django displays the debug screen instead.
When DEBUG = False, Django tries to display the corresponding error template, which could be customized for the application.
The KeyError appears when Django fails to fully process or display the error page. This often happens if the error template triggers some data processing that may not be available when there's a server error.
In my case, my error template was inheriting the base template, which had dynamic content like user data and menus. Replacing the error template with a simple, static version that doesn’t require much dynamic data resolved the KeyError issue for me.

I hope this helps others who may be encountering the same problem and saves time for the Sentry team!

Cheers

sl0thentr0py commented 2 weeks ago

okay I'm investigating this now, sorry for the delay.

I agree with @lbillon that this doesn't seem to be Sentry's problem. In the examples @NicoEdge5 provided, the key request is missing in the context inside django-cms here while rendering the django error template.

@lbillon are you also using django-cms? And can either or both of you provide your error templates so I can reproduce the problem?

sl0thentr0py commented 2 weeks ago

ok I know what's happening in @NicoEdge5's case now

ALLOWED_HOSTS doesn't have that particular host from your error so it bails early in the pipeline and django tried to render the error page.
since it bails early, django.template.context_processors.request doesn't run so the context doesn't have the request key
the error page that is being rendered uses some classytags stuff which fails when accessing that context key

getsentry / sentry-python