"Query not supported" when running on django-nonrel / AppEngine

[tjohns]

I'm trying to get a basic django-rest-framework app up and running using AppEngine, via django-nonrel.

When I try to get the tutorial running, navigating to /users or /groups gives me the following error: "This query is not supported by the database."

Full stack trace below:

Environment:

Request Method: GET
Request URL: http://localhost:8000/users/

Django Version: 1.4.5
Python Version: 2.7.3
Installed Applications:
('django.contrib.admin',
 'django.contrib.auth',
 'django.contrib.contenttypes',
 'django.contrib.sessions',
 'django.contrib.staticfiles',
 'djangoappengine',
 'rest_framework',
 'autoload',
 'dbindexer',
 'djangotoolbox')
Installed Middleware:
('autoload.middleware.AutoloadMiddleware',
 'django.middleware.common.CommonMiddleware',
 'django.contrib.sessions.middleware.SessionMiddleware',
 'django.contrib.auth.middleware.AuthenticationMiddleware')

Traceback:
File ".../myapp/django/core/handlers/base.py" in get_response
  111.                         response = callback(request, *callback_args, **callback_kwargs)
File ".../myapp/rest_framework/viewsets.py" in view
  78.             return self.dispatch(request, *args, **kwargs)
File ".../myapp/django/views/decorators/csrf.py" in wrapped_view
  77.         return view_func(*args, **kwargs)
File ".../myapp/rest_framework/views.py" in dispatch
  327.             response = self.handle_exception(exc)
File ".../myapp/rest_framework/views.py" in dispatch
  324.             response = handler(request, *args, **kwargs)
File ".../myapp/rest_framework/mixins.py" in list
  96.         return Response(serializer.data)
File ".../myapp/rest_framework/serializers.py" in data
  499.                 self._data = [self.to_native(item) for item in obj]
File ".../myapp/rest_framework/serializers.py" in to_native
  306.             value = field.field_to_native(obj, field_name)
File ".../myapp/rest_framework/relations.py" in field_to_native
  148.                 return [self.to_native(item) for item in value.all()]
File ".../myapp/django/db/models/query.py" in _result_iter
  118.                 self._fill_cache()
File ".../myapp/django/db/models/query.py" in _fill_cache
  892.                     self._result_cache.append(self._iter.next())
File ".../myapp/django/db/models/query.py" in iterator
  291.         for row in compiler.results_iter():
File ".../myapp/djangotoolbox/db/basecompiler.py" in results_iter
  335.         results = self.build_query(fields).fetch(
File ".../myapp/djangotoolbox/db/basecompiler.py" in build_query
  426.         self.check_query()
File ".../myapp/djangotoolbox/db/basecompiler.py" in check_query
  407.             raise DatabaseError("This query is not supported by the database.")

Exception Type: DatabaseError at /users/
Exception Value: This query is not supported by the database.

[tjohns]

Specifically, here's the method in django-nonrel that's generating the error:

djangotoolbox/db/basecompiler.py:

    def check_query(self):
        """
        Checks if the current query is supported by the database.

        In general, we expect queries requiring JOINs (many-to-many
        relations, abstract model bases, or model spanning filtering),
        using DISTINCT (through `QuerySet.distinct()`, which is not
        required in most situations) or using the SQL-specific
        `QuerySet.extra()` to not work with nonrel back-ends.
        """
        if (len([a for a in self.query.alias_map if
                 self.query.alias_refcount[a]]) > 1 or
            self.query.distinct or self.query.extra or self.query.having):
            raise DatabaseError("This query is not supported by the database.")

[kevin-brown]

It sounds like the models have foreign-key relationships, which Django Rest Framework is following. Django-nonrel does not support them (by design) and throws an exception when they are used.

[tomchristie]

As @kevin-brown notes this'll be to do with the types of model relationships/lookups that GAE supports.

You're not going to be able to simply port the generic views across to GAE without taking into account the differences in GAE datastore vs relational databases. I'd strongly suggest working with APIView and regular Serializers if you want to use app engine, so that you're dealing with any model behaviour explicitly.

I'm going to close this ticket, as we don't claim support for GAE - I'd love for someone to document any considerations when running with non-rel, or packaging up any differences that might be required to make the generic views and/or model serializers work more seamlessly with it, but if that does happen it needs to be someone taking that on independently (with us linking to them from the main docs), rather than as part of the core REST framework project.

[tjohns]

Thanks for clarifying.

As an aside, for anyone else who comes across this ticket: I did get things working with my own custom models. Specifically, the Polls/Choices models listed in the django-rest-framework tutorial work without any changes.

It seems it was just the Users/Groups models provided by the framework that were giving me trouble.

Given that most apps using django-nonrel are going to avoid using ForeignKey in their own models (because of GAE's restrictions), this doesn't seem to be a huge issue.

[thochi]

ForeignKeys are not the problem. They are working just fine. As stated in the docs, many to many relationships and joins are not supported by django nonrel. I could work around this issue by omitting the groups relation with a custom serializer:

class UserSerializer(serializers.ModelSerializer):
    class Meta:
        model = User
        fields = ('id', 'username', 'email', 'password', 'first_name', 'last_name')

[arnm]

Thanks thochi! Work greak

[gminds]

changing the fields worked like charm. Thanks thochi

[kkostenko]

Hey folks,

I am facing the same or similar issue where django 1.6 non-rel do not support a query below. It looks like I need a custom serializer but current restframework requires django 1.8 http://www.django-rest-framework.org/#requirements. I am wondering where can I get rest_framework for 1.6 non-rel? Perhaps there is some other workaround?

super(FindResultsDataView, self).get_queryset().filter(feature__name = 'name') *** DatabaseError: This query is not supported by the database.