cache response of DRF view?

[palvarezcordoba]

Hi, I'm wondering if there's any convenient way to cache the response of a Django Rest framework endpoint. For what I've seen, there's no easy way.

I did this to use cacheops to cache the response, invalidating it if the model Project was created/changed/deleted:

def list(self, request, *args, **kwargs):
        queryset = self.filter_queryset(self.get_queryset())
        page = self.paginate_queryset(queryset)

        @cached_as(
            Project,
            extra=lambda: (
                request.user.user_type,
                request.user.country,
                request.user.currency_token,
                request.user.company.is_intracommunity_operator if request.user.company else False,
            ),
        )
        def _list():
            if page is not None:
                serializer = self.get_serializer(page, many=True)
                return serializer.data
            serializer = self.get_serializer(queryset, many=True)
            return serializer.data
        data = _list()
        if page is not None:
            return self.get_paginated_response(data)
        else:
            return Response(data)

I needed a fast solution so this works for me, at this time. But I can't keep doing this all over the project if I need do cache more endpoints.

[reallyBhatti]

Did you figure out a simpler solution to this? I am currently trying to do something similar and I've got numerous APIs. Doing this all over the project is not feasible.

[campenr]

Does the View caching from the README not fit your needs? Per the docs:

from cacheops import cached_view_as

@cached_view_as(News)
def news_index(request):
    # ...
    return render(...)

Or in your case

from cacheops import cached_view_as

@cached_view_as(Project)
def list(self, request, *args, **kwargs):
    # ...

This decorator also takes the extra argument so you can pass the same lambda as in your example if needed.

[palvarezcordoba]

@campenr I don't know why I didn't do it that way. I'll try that today/tomorrow and will say if worked or not.

@reallyBhatti Sorry, I didn't read github notifications. I'm still doing it very similar to my original comment. You can try to do it the way @campenr said

[Suor]

I never used DRF myself but saw people used some subclasses to marry it with cacheops.

A couple of notes on original code sample:

1. One might pass queryset to @cached_as() and @cached_view_as(), this will probably result in a more granular invalidation than simply passing Project.
2. extra does not need to be a lambda, it could be the tuple directly.

[modbender]

@Suor I wanted to use cached_view_as, but the request received inside REST API view is of type rest_framework.request.Request

But you have a assert at https://github.com/Suor/django-cacheops/blob/eb8218b5fc602fa8d89a3fc4317eb9482f42b71d/cacheops/utils.py#L106 checking if the request is of type HTTPRequest which is raising error AssertionError: A view should be passed with HttpRequest as first argument

BTW this is the code I'm using:

from rest_framework import viewsets

from cacheops import cached_view_as

class BaseModelViewSet(viewsets.ModelViewSet):

    def list(self, request, *args, **kwargs):
        qs = self.get_queryset()

        @cached_view_as(qs)
        def _cached_list(request, *args, **kwargs):
            return super(BaseModelViewSet, self).list(request, *args, **kwargs)

        response = _cached_list(request, *args, **kwargs)

        return response

    class Meta:
        abtract = True

EDIT 1: I locally edited the code and commented that line, but still I'm seeing this error:

Traceback (most recent call last):
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\core\handlers\exception.py", line 55, in inner
    response = get_response(request)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\core\handlers\base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\views\decorators\csrf.py", line 54, in wrapped_view        
    return view_func(*args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 480, in raise_uncaught_exception   
    raise exc
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "E:\Project\django\site-py\api\views\base.py", line 15, in list
    response = _cached_list(request, *args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\cacheops\utils.py", line 111, in wrapper
    return cached_func(request, *args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\cacheops\query.py", line 132, in wrapper
    result = func(*args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\funcy\funcs.py", line 108, in <lambda>
    pair = lambda f, g: lambda *a, **kw: f(g(*a, **kw))
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\cacheops\utils.py", line 97, in force_render
    response.render()
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\template\response.py", line 114, in render
    self.content = self.rendered_content
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\response.py", line 55, in rendered_content
    assert renderer, ".accepted_renderer not set on Response"
AssertionError: .accepted_renderer not set on Response

EDIT 2: Using cache_page works perfectly fine, you might have to change some asserts to the way cache_page checks for attribute

from django.views.decorators.cache import cache_page https://github.com/django/django/blob/main/django/views/decorators/cache.py

[Suor]

One of the things @cached_view_as() does is calling .render() method on a response, this is done to fill it in properly before serializing and sending to cache. This might be interfering with what DRF expects, i.e. this .accepted_renderer doesn't look like a part of Django. It would be somewhat bit weird for cacheops to know such DRF internals, so the proper place to glue such things is the code using both. I would maybe ignore that being weird if this request was very common or if I myself used DRF and were familiar with its internals. For now though I don't think this belongs to cacheops code.

There are ways for you to make it work though. The most trivial is to cache only queryset not the list or whatever:

class BaseModelViewSet(viewsets.ModelViewSet):
    def get_queryset(self):
        return super().get_queryset().cache()

    class Meta:
        abtract = True

Similarly trivial will be just setting caching for needed models automatically in the CACHEOPS setting.

If you insist on caching the list then you way try a more generic purpose @cached_as() instead:

class BaseModelViewSet(viewsets.ModelViewSet):
    def list(self, request, *args, **kwargs):
        qs = self.get_queryset()

        # All arguments to this will be used in a cache key, HttpRequest will be turned into uri and
        # some other transformations are made, but be aware of what you are passing here. Anything
        # not varying the result should not go.
        @cached_as(qs)
        def _cached_list(request, *args, **kwargs):
            # This should not return any lazy thing like djangos TemplateResponse
            return super(BaseModelViewSet, self).list(request, *args, **kwargs)

        return _cached_list(request, *args, **kwargs)

    class Meta:
        abtract = True

@cached_view_as() is a shallow enough wrapper for @cached_as() doing only a couple of things

[Suor]

This should do it for now. Please reopen if none of the recipes works though.

[modbender]

@Suor The problem with using @cached_as the way you mentioned is, it gives this error:

Traceback (most recent call last):
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\core\handlers\exception.py", line 55, in inner
    response = get_response(request)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\core\handlers\base.py", line 197, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\views\decorators\csrf.py", line 54, in wrapped_view        
    return view_func(*args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\viewsets.py", line 125, in view
    return self.dispatch(request, *args, **kwargs)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 509, in dispatch
    response = self.handle_exception(exc)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 469, in handle_exception
    self.raise_uncaught_exception(exc)
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 480, in raise_uncaught_exception   
    raise exc
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\views.py", line 506, in dispatch
    response = handler(request, *args, **kwargs)
  File "E:\Project\django\site-py\api\views\base.py", line 18, in list
    return _cached_list(request, *args, **kwargs)
  File "E:\Project\django\site-py\cacheops\query.py", line 110, in wrapper
    cache_thing(prefix, cache_key, result, cond_dnfs, timeout, dbs=dbs,
  File "E:\Project\django\site-py\cacheops\getset.py", line 47, in cache_thing
    settings.CACHEOPS_SERIALIZER.dumps(data),
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\rest_framework\response.py", line 95, in __getstate__
    state = super().__getstate__()
  File "C:\Users\User\anaconda3\envs\site-py\lib\site-packages\django\template\response.py", line 60, in __getstate__
    raise ContentNotRenderedError(
django.template.response.ContentNotRenderedError: The response content must be rendered before it can be pickled.

That's why I switched to @cached_view_as hoping it would solve the issue.

The highlightable point of django decorator cache_page is that it's using CacheMiddleware

[Suor]

So it's indeed a TemplateResponse, you need to render it the same way @cached_view_as() does.

[Suor]

There is an unfinished PR about this #453