Server error page won't render because of missing context

[amickan]

If a view throws a 500 error, the 500.html should get rendered. We recently updated the template to inherit from base.html, and now it will not render anymore because it is missing context variables (the 500 view is by default passed an empty context).

I'm unsure if we should update the 500 view and add the missing context or if we should go back to not inheriting from base.html for the error views?

[jmsmkn]

This has already been fixed in #3372.

[amickan]

Hm I am on main and I still get this error in tests (when I intentionally mess things up so that I get a server error).

[jmsmkn]

Really? It should be covered by the test server settings. I just tried by adding a 1/0 in L37 of app/grandchallenge/core/views.py and tests/core_tests/test_views.py::test_main returns ZeroDivisionError: division by zero as expected. If I do the same on ac5d9db21ae1841ec617014f8b83f1679464e104 I get the old tests/core_tests/test_views.py::test_main - django.template.base.VariableDoesNotExist: Failed lookup for key [about_page_url] in [{'True': True, 'False': False, 'None': None}].

Can you provide a short diff that replicates it?

[amickan]

If I run a test for a view that is missing a template, I still get django.template.base.VariableDoesNotExist: Failed lookup for key [about_page_url] in [{'True': True, 'False': False, 'None': None}]. For example, run tests/evaluation_tests/test_views.py::test_ground_truth_permissions and rename one of the groundtruth view templates beforehand.

[jmsmkn]

I still cannot replicate it using either my native python setup or through docker:

(grand-challenge-org-py3.12) ➜  app git:(main) ✗ git rev-parse --short HEAD
aff5ecc6c
(grand-challenge-org-py3.12) ➜  app git:(main) ✗ git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add/rm <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        deleted:    grandchallenge/core/templates/home.html

no changes added to commit (use "git add" and/or "git commit -a")
(grand-challenge-org-py3.12) ➜  app git:(main) ✗ poetry run pytest tests/core_tests/test_views.py::test_main -n1 --reuse-db 
=============================================================================================================================== test session starts ===============================================================================================================================
platform darwin -- Python 3.12.3, pytest-8.2.2, pluggy-1.5.0
cachedir: /tmp/.pytest_cache
Using --randomly-seed=2985794938
django: version: 4.2.13, settings: tests.settings (from ini)
rootdir: /Users/jamesmeakin/Documents/GitHub/grand-challenge.org/app
configfile: pytest.ini
plugins: cov-5.0.0, playwright-0.5.0, randomly-3.15.0, rerunfailures-14.0, Faker-25.8.0, anyio-4.4.0, django-4.8.0, base-url-2.1.0, mock-3.14.0, xdist-3.6.1
1 worker [1 item]      
F                                                                                                                                                                                                                                                                           [100%]
==================================================================================================================================== FAILURES =====================================================================================================================================
____________________________________________________________________________________________________________________________________ test_main ____________________________________________________________________________________________________________________________________
[gw0] darwin -- Python 3.12.3 /Users/jamesmeakin/Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/bin/python

client = <django.test.client.Client object at 0x14dc1b890>

    @pytest.mark.django_db
    def test_main(client):
        url = reverse("home")
>       response = client.get(url)

client     = <django.test.client.Client object at 0x14dc1b890>
url        = 'https://testserver/'

tests/core_tests/test_views.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/test/client.py:927: in get
    response = super().get(path, data=data, secure=secure, headers=headers, **extra)
        __class__  = <class 'django.test.client.Client'>
        data       = None
        extra      = {}
        follow     = False
        headers    = None
        path       = 'https://testserver/'
        secure     = False
        self       = <django.test.client.Client object at 0x14dc1b890>
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/test/client.py:457: in get
    return self.generic(
        data       = {}
        extra      = {}
        headers    = None
        path       = 'https://testserver/'
        secure     = False
        self       = <django.test.client.Client object at 0x14dc1b890>
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/test/client.py:609: in generic
    return self.request(**r)
        content_type = 'application/octet-stream'
        data       = b''
        extra      = {'QUERY_STRING': ''}
        headers    = None
        method     = 'GET'
        parsed     = ParseResult(scheme='https', netloc='testserver', path='/', params='', query='', fragment='')
        path       = 'https://testserver/'
        query_string = ''
        r          = {'PATH_INFO': '/', 'QUERY_STRING': '', 'REQUEST_METHOD': 'GET', 'SERVER_PORT': '80', ...}
        secure     = False
        self       = <django.test.client.Client object at 0x14dc1b890>
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/test/client.py:891: in request
    self.check_exception(response)
        data       = {'context': [[{'True': True, 'False': False, 'None': None}, {'csrf_token': <SimpleLazyObject: '18siI505WUXjJEVf2Falaug...%}...">, <Template template_string="{% load url %}{% lo...">, <Template template_string="{% load static %}{...">, ...]}
        environ    = {'CSRF_COOKIE': 'iodZj6EcZB5L7MmLxHJKPCnTpW8foqDg', 'CSRF_COOKIE_NEEDS_UPDATE': False, 'HTTP_COOKIE': '', 'PATH_INFO': '/', ...}
        exception_uid = 'request-exception-5625322240'
        on_template_render = functools.partial(<function store_rendered_templates at 0x1287c9940>, {'templates': [<Template template_string="{% ext...TextNode: '\n\n    '>, <IfNode>, <TextNode: '\n\n'>]>}, {'set_timezone_url': 'https://testserver/api/v1/timezone/'}]]})
        request    = {'PATH_INFO': '/', 'QUERY_STRING': '', 'REQUEST_METHOD': 'GET', 'SERVER_PORT': '80', ...}
        response   = <HttpResponse status_code=500, "text/html; charset=utf-8">
        self       = <django.test.client.Client object at 0x14dc1b890>
        signal_uid = 'template-render-5625322240'
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/test/client.py:738: in check_exception
    raise exc_value
        _          = <traceback object at 0x14f6e9100>
        exc_value  = TemplateDoesNotExist('home.html')
        response   = <HttpResponse status_code=500, "text/html; charset=utf-8">
        self       = <django.test.client.Client object at 0x14dc1b890>
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/core/handlers/exception.py:55: in inner
    response = get_response(request)
        get_response = <bound method BaseHandler._get_response of <django.test.client.ClientHandler object at 0x14dc1b770>>
        request    = <WSGIRequest: GET '/'>
        response   = <HttpResponse status_code=500, "text/html; charset=utf-8">
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/core/handlers/base.py:220: in _get_response
    response = response.render()
        callback   = <function View.as_view.<locals>.view at 0x14e642de0>
        callback_args = ()
        callback_kwargs = {}
        middleware_method = <bound method RequireStaffAndSuperuser2FAMiddleware.process_view of <RequireStaffAndSuperuser2FAMiddleware get_response=convert_exception_to_response.<locals>.inner>>
        request    = <WSGIRequest: GET '/'>
        response   = None
        self       = <django.test.client.ClientHandler object at 0x14dc1b770>
        wrapped_callback = <function View.as_view.<locals>.view at 0x14f50a2a0>
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/template/response.py:114: in render
    self.content = self.rendered_content
        retval     = <TemplateResponse status_code=200, "text/html; charset=utf-8">
        self       = <TemplateResponse status_code=200, "text/html; charset=utf-8">
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/template/response.py:90: in rendered_content
    template = self.resolve_template(self.template_name)
        self       = <TemplateResponse status_code=200, "text/html; charset=utf-8">
../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/template/response.py:72: in resolve_template
    return select_template(template, using=self.using)
        self       = <TemplateResponse status_code=200, "text/html; charset=utf-8">
        template   = ['home.html']
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

template_name_list = ['home.html'], using = None

    def select_template(template_name_list, using=None):
        """
        Load and return a template for one of the given names.

        Try names in order and return the first template found.

        Raise TemplateDoesNotExist if no such template exists.
        """
        if isinstance(template_name_list, str):
            raise TypeError(
                "select_template() takes an iterable of template names but got a "
                "string: %r. Use get_template() if you want to load a single "
                "template by name." % template_name_list
            )

        chain = []
        engines = _engine_list(using)
        for template_name in template_name_list:
            for engine in engines:
                try:
                    return engine.get_template(template_name)
                except TemplateDoesNotExist as e:
                    chain.append(e)

        if template_name_list:
>           raise TemplateDoesNotExist(", ".join(template_name_list), chain=chain)
E           django.template.exceptions.TemplateDoesNotExist: home.html

chain      = [TemplateDoesNotExist('home.html')]
engine     = <django.template.backends.django.DjangoTemplates object at 0x14f4a9010>
engines    = [<django.template.backends.django.DjangoTemplates object at 0x14f4a9010>]
template_name = 'home.html'
template_name_list = ['home.html']
using      = None

../../../../Library/Caches/pypoetry/virtualenvs/grand-challenge-org-DgCYUgrQ-py3.12/lib/python3.12/site-packages/django/template/loader.py:47: TemplateDoesNotExist
============================================================================================================================= short test summary info =============================================================================================================================
FAILED tests/core_tests/test_views.py::test_main - django.template.exceptions.TemplateDoesNotExist: home.html
================================================================================================================================ 1 failed in 4.38s ================================================================================================================================
(grand-challenge-org-py3.12) ➜  app git:(main) ✗ docker compose run --rm celery_worker_evaluation pytest tests/core_tests/test_views.py::test_main -n1 --reuse-db
[+] Creating 5/0
 ✔ Container grand-challengeorg-minio.localhost-1  Running                                                                                                                                                                                                                    0.0s 
 ✔ Container grand-challengeorg-postgres-1         Running                                                                                                                                                                                                                    0.0s 
 ✔ Container grand-challengeorg-registry-1         Running                                                                                                                                                                                                                    0.0s 
 ✔ Container grand-challengeorg-redis-1            Running                                                                                                                                                                                                                    0.0s 
 ✔ Container grand-challengeorg-web-1              Running                                                                                                                                                                                                                    0.0s 
[+] Running 1/1
 ✔ Container grand-challengeorg-postgres-1  Healthy                                                                                                                                                                                                                           0.5s 
=============================================================================================================================== test session starts ===============================================================================================================================
platform linux -- Python 3.11.9, pytest-8.2.2, pluggy-1.5.0
cachedir: /tmp/.pytest_cache
Using --randomly-seed=1930612647
django: version: 4.2.13, settings: tests.settings (from ini)
rootdir: /app
configfile: pytest.ini
plugins: anyio-4.4.0, Faker-25.8.0, randomly-3.15.0, cov-5.0.0, xdist-3.6.1, django-4.8.0, rerunfailures-14.0, mock-3.14.0, base-url-2.1.0, playwright-0.5.0
1 worker [1 item]      
F                                                                                                                                                                                                                                                                           [100%]
==================================================================================================================================== FAILURES =====================================================================================================================================
____________________________________________________________________________________________________________________________________ test_main ____________________________________________________________________________________________________________________________________
[gw0] linux -- Python 3.11.9 /opt/poetry/.venv/bin/python

client = <django.test.client.Client object at 0x7fffb51d4810>

    @pytest.mark.django_db
    def test_main(client):
        url = reverse("home")
>       response = client.get(url)

client     = <django.test.client.Client object at 0x7fffb51d4810>
url        = 'https://testserver/'

tests/core_tests/test_views.py:16: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/opt/poetry/.venv/lib/python3.11/site-packages/django/test/client.py:927: in get
    response = super().get(path, data=data, secure=secure, headers=headers, **extra)
        __class__  = <class 'django.test.client.Client'>
        data       = None
        extra      = {}
        follow     = False
        headers    = None
        path       = 'https://testserver/'
        secure     = False
        self       = <django.test.client.Client object at 0x7fffb51d4810>
/opt/poetry/.venv/lib/python3.11/site-packages/django/test/client.py:457: in get
    return self.generic(
        data       = {}
        extra      = {}
        headers    = None
        path       = 'https://testserver/'
        secure     = False
        self       = <django.test.client.Client object at 0x7fffb51d4810>
/opt/poetry/.venv/lib/python3.11/site-packages/django/test/client.py:609: in generic
    return self.request(**r)
        content_type = 'application/octet-stream'
        data       = b''
        extra      = {'QUERY_STRING': ''}
        headers    = None
        method     = 'GET'
        parsed     = ParseResult(scheme='https', netloc='testserver', path='/', params='', query='', fragment='')
        path       = 'https://testserver/'
        query_string = ''
        r          = {'PATH_INFO': '/', 'QUERY_STRING': '', 'REQUEST_METHOD': 'GET', 'SERVER_PORT': '80', ...}
        secure     = False
        self       = <django.test.client.Client object at 0x7fffb51d4810>
/opt/poetry/.venv/lib/python3.11/site-packages/django/test/client.py:891: in request
    self.check_exception(response)
        data       = {'context': [[{'True': True, 'False': False, 'None': None}, {'csrf_token': <SimpleLazyObject: 'hdHf12Z9SgOG1OtbIHY7s0W...%}...">, <Template template_string="{% load url %}{% lo...">, <Template template_string="{% load static %}{...">, ...]}
        environ    = {'CSRF_COOKIE': '6pOXD2m14r3KFnQnieKXQ6UexMPMtqQ3', 'CSRF_COOKIE_NEEDS_UPDATE': False, 'HTTP_COOKIE': '', 'PATH_INFO': '/', ...}
        exception_uid = 'request-exception-140736244164416'
        on_template_render = functools.partial(<function store_rendered_templates at 0x7fffd6ff16c0>, {'templates': [<Template template_string="{% ...TextNode: '\n\n    '>, <IfNode>, <TextNode: '\n\n'>]>}, {'set_timezone_url': 'https://testserver/api/v1/timezone/'}]]})
        request    = {'PATH_INFO': '/', 'QUERY_STRING': '', 'REQUEST_METHOD': 'GET', 'SERVER_PORT': '80', ...}
        response   = <HttpResponse status_code=500, "text/html; charset=utf-8">
        self       = <django.test.client.Client object at 0x7fffb51d4810>
        signal_uid = 'template-render-140736244164416'
/opt/poetry/.venv/lib/python3.11/site-packages/django/test/client.py:738: in check_exception
    raise exc_value
        _          = <traceback object at 0x7fffb5edcec0>
        exc_value  = TemplateDoesNotExist('home.html')
        response   = <HttpResponse status_code=500, "text/html; charset=utf-8">
        self       = <django.test.client.Client object at 0x7fffb51d4810>
/opt/poetry/.venv/lib/python3.11/site-packages/django/core/handlers/exception.py:55: in inner
    response = get_response(request)
        get_response = <bound method BaseHandler._get_response of <django.test.client.ClientHandler object at 0x7fffb5b9e310>>
        request    = <WSGIRequest: GET '/'>
        response   = <HttpResponse status_code=500, "text/html; charset=utf-8">
/opt/poetry/.venv/lib/python3.11/site-packages/django/core/handlers/base.py:220: in _get_response
    response = response.render()
        callback   = <function View.as_view.<locals>.view at 0x7fffd5a7e480>
        callback_args = ()
        callback_kwargs = {}
        middleware_method = <bound method RequireStaffAndSuperuser2FAMiddleware.process_view of <RequireStaffAndSuperuser2FAMiddleware get_response=convert_exception_to_response.<locals>.inner>>
        request    = <WSGIRequest: GET '/'>
        response   = None
        self       = <django.test.client.ClientHandler object at 0x7fffb5b9e310>
        wrapped_callback = <function View.as_view.<locals>.view at 0x7fffb5575f80>
/opt/poetry/.venv/lib/python3.11/site-packages/django/template/response.py:114: in render
    self.content = self.rendered_content
        retval     = <TemplateResponse status_code=200, "text/html; charset=utf-8">
        self       = <TemplateResponse status_code=200, "text/html; charset=utf-8">
/opt/poetry/.venv/lib/python3.11/site-packages/django/template/response.py:90: in rendered_content
    template = self.resolve_template(self.template_name)
        self       = <TemplateResponse status_code=200, "text/html; charset=utf-8">
/opt/poetry/.venv/lib/python3.11/site-packages/django/template/response.py:72: in resolve_template
    return select_template(template, using=self.using)
        self       = <TemplateResponse status_code=200, "text/html; charset=utf-8">
        template   = ['home.html']
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _

template_name_list = ['home.html'], using = None

    def select_template(template_name_list, using=None):
        """
        Load and return a template for one of the given names.

        Try names in order and return the first template found.

        Raise TemplateDoesNotExist if no such template exists.
        """
        if isinstance(template_name_list, str):
            raise TypeError(
                "select_template() takes an iterable of template names but got a "
                "string: %r. Use get_template() if you want to load a single "
                "template by name." % template_name_list
            )

        chain = []
        engines = _engine_list(using)
        for template_name in template_name_list:
            for engine in engines:
                try:
                    return engine.get_template(template_name)
                except TemplateDoesNotExist as e:
                    chain.append(e)

        if template_name_list:
>           raise TemplateDoesNotExist(", ".join(template_name_list), chain=chain)
E           django.template.exceptions.TemplateDoesNotExist: home.html

chain      = [TemplateDoesNotExist('home.html')]
engine     = <django.template.backends.django.DjangoTemplates object at 0x7fffb57bd2d0>
engines    = [<django.template.backends.django.DjangoTemplates object at 0x7fffb57bd2d0>]
template_name = 'home.html'
template_name_list = ['home.html']
using      = None

/opt/poetry/.venv/lib/python3.11/site-packages/django/template/loader.py:47: TemplateDoesNotExist
============================================================================================================================= short test summary info =============================================================================================================================
FAILED tests/core_tests/test_views.py::test_main - django.template.exceptions.TemplateDoesNotExist: home.html
=============================================================================================================================== 1 failed in 56.29s ================================================================================================================================

[jmsmkn]

Ah wait, I missed the bit about the ground truth view, it does work for the home page.

[jmsmkn]

The handler500 was missing from the challenge subdomain root urls, so was still failing on challenge views. Easy fix.

[jmsmkn]

Confirmed that this solves the issue:

FAILED tests/evaluation_tests/test_views.py::test_ground_truth_permissions - django.template.exceptions.TemplateDoesNotExist: evaluation/evaluationgroundtruth_form.html

[amickan]

Thanks for fixing!