search

lupinia / awi

Full Lupinia/Awi website, built with Django
https://www.lupinia.net/
2 stars 1 forks source link

Context processors missing from error views #90

Closed lupinia closed 1 year ago

lupinia commented 1 year ago

Core problem that needs solving: In all of the error views (403, 404, and 500), context processors are not executed, so any extra context provided by them is missing entirely from the templates. This has been a long-standing problem, pretty much since the beginning, and was always a blocker to #26, but otherwise more of an annoyance than anything else. However, it came up in the context of trying to finish #88, and it's now a blocker to #61.

Trying to figure out why this is a problem has led me down a rabbit hole of circular research, so I'll try to document my findings in as much detail as possible, for the benefit of anyone who might stumble across this in the future.

For these error views, I'm not using class-based views, I'm using function views. When I first developed them, this was the approach recommended by the Django documentation, and it continues to be the most straightforward way to return a response using the special error handler objects (django.http.HttpResponseNotFound, django.http.HttpResponseServerError, and django.http.HttpResponseForbidden). These response objects take a content parameter, which is supposed to be a rendered template. And the template.render() method, in turn, takes a context argument. So, in my error views, I'm setting up a template object using django.template.loader.get_template, stored to a variable named template, and then creating a dictionary named context, which are then assembled together in the error response object as content=template.render(context) The end result looks something like this:

from django.template import loader

def not_found(request):
    # Irrelevant parts removed
    template = loader.get_template('404.html')
    context = {'old_url':request.path,}
    # Irrelevant parts removed
    return HttpResponseNotFound(content=template.render(context), content_type='text/html; charset=utf-8')

When searching for a solution to missing context processors in Django function views more generally, the consensus is that the context object needs to be initialized not as a dictionary, but as a django.template.RequestContext object. The documentation appears to support this. However, if I change context in the above code to be a RequestContext object, I get a TypeError exception: context must be a dict rather than RequestContext.

Searching for this exception yields seemingly contradictory information: As of Django 1.11, passing a Context or RequestContext object directly to template rendering is no longer supported, it must be a dictionary. Which makes sense, but how am I supposed to get all the extra execution that RequestContext performs? Is that now happening inside template renderers? Well, no, not as far as I can tell. The official solutions all mention using django.shortcuts.render, which "automatically uses RequestContext". How, exactly? It's unclear. However, what IS clear is that this render shortcut returns an HttpResponse object, which won't work because I'm trying to pass the results into a response object.

Also, the documentation appears to be incorrect, not just for the version I'm using, but for all versions; even for the newest version (as of this writing), the official documentation STILL says "Call the Template object's render() method with a Context to 'fill' the template": https://docs.djangoproject.com/en/4.2/ref/templates/api/#rendering-a-context But if you follow these instructions as-written, as I am, you will get the aforementioned error.

I did find a clue in this ticket: https://code.djangoproject.com/ticket/28491 Apparently, django.template.loader does not return an object of type django.template.Template, which is what the documentation is for. It returns a different type of object with a different .render() method, which does not accept a Context object, even though the lower-level Template object does.

A big part of the problem here is the naming conventions, since there are multiple layers of these objects that all have identical or similar names. And while I've been referring to them by their full paths in this ticket, that's only to keep myself from getting confused while trying to sort all this out. No one - myself included - actually uses them by those full paths, because that's not how Python works (or, well, I guess you technically could do it that way, but why?). So that makes this problem effectively unsearchable. However, the specific description for this specific django.template.backends.base.Template.render() method - which is different from the django.template.Template.render() method in fundamentally incompatible ways - indicates that it takes two arguments, context and request, instead of just one like the much more thorough documentation that everyone references.

And that's the solution: If you're using function-based views in Django (perhaps working with legacy code that dates back to the earliest days of Django 1.3, which is where I started), and your context processors are missing, and you're using a Template object created by django.templates.loader, and trying to fix it in all the ways every search tells you to fix it results in exceptions like context must be a dict rather than RequestContext, the actual solution is to pass the context dictionary AND the request object to your Template.render() method.

lupinia commented 1 year ago

Upon further research, it's possible that this bug in my website specifically is a lot newer than I thought. I was certain that it has always been this way, but perhaps not. The aforementioned change to passing a dictionary instead of a Context object to template.render() in these error views was introduced in this commit (for issue #45): https://github.com/lupinia/awi/commit/680d07ba1e7af217aac522e2e371ffd0dc6b30c3

Essentially, what happened here is that Django fundamentally changed how template rendering works between 1.8 and 1.11, didn't properly document the extent of that change, and all searches trying to find how to fix the new error yielded an incomplete solution for what was actually a very different problem, even though all the object names are the same. So, if you just do what every StackOverflow question and Django's own documentation say to do to fix this exception (just convert the Context object to a plain dictionary and everything's fine!), you'll lose all your context processors, because you ALSO have to add a new request parameter to Template.render(), IF and only if you're initializing your Template object using django.templates.loader - which returns an object of a different class from what shows up if you try to look up the documentation for Template.render().

At the time of writing this comment, I've spent about 9 hours on this, so I really hope this issue helps someone else avoid the same problem someday. (Given the current sad state of Google search results, there may be hundreds of posts just like this one floating around in the aether that just don't show up anymore, but this is my bug tracker, not my blog, so that's a rant for another day).

EDIT: To clarify, this actually was always broken, it just used to be broken in a slightly different way, and had I fixed it before the 1.11 update, I might have caught this sooner. With the old pre-1.11 code, I was initializing a raw Context object, and passing that to Template.render(); the way to fix it at that time would've been to switch this to a RequestContext object. Had I done that, the conversion to passing a plain dictionary to Template.render() might've made me ask "wait, what about the request object to set up the context processors?" and dig deeper. I definitely don't think it would've been any easier to find the correct answer, though, given the name collisions for every single piece of this puzzle.