Subqueries fail silently

[Wilfred]

Taken from this mailing list discussion: http://groups.google.com/group/django-non-relational/browse_thread/thread/c006d67444bb28f7

Given the models:

from django.db import models

class Person(models.Model):
    name = models.CharField(max_length=200)

class Car(models.Model):
    name = models.CharField(max_length=200)
    owner = models.ForeignKey(Person)

class Wheel(models.Model):
    part_of = models.ForeignKey(Car)

Subqueries return no results, even when there should be

In [1]: from test_app.models import Person, Car, Wheel

In [2]: p = Person.objects.create(name='bob')

In [3]: c1 = Car.objects.create(name='mini', owner=p)

In [4]: c2 = Car.objects.create(name='landrover', owner=p)

In [5]: w = Wheel.objects.create(part_of=c2)

In [6]: cars = Car.objects.filter(owner=p)

In [7]: cars
Out[7]: [<Car: Car object>, <Car: Car object>]

In [8]: Wheel.objects.filter(part_of__in=cars) # cars is a QuerySet
Out[8]: []

In [9]: Wheel.objects.filter(part_of__in=list(cars)) # force QuerySet to list
Out[9]: [<Wheel: Wheel object>]

[aburgel]

do any of the non-rel backends support subqueries? i know appengine doesn't, so if the others also don't, we could raise an exception.

[jonashaag]

Wilfred, what happens if you put a len(...) around input number 8? Maybe Django swallows exceptions (if an exception happens in the backend, Django simply returns the empty list)

[Wilfred]

In [8]: len(Wheel.objects.filter(part_of__in=cars))
---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
/home/wilfred/work/gap2/<ipython-input-8-072b023c1483> in <module>()
----> 1 len(Wheel.objects.filter(part_of__in=cars))

/home/wilfred/work/gap2/django/db/models/query.py in __len__(self)
     81                 self._result_cache = list(self._iter)
     82             else:
---> 83                 self._result_cache = list(self.iterator())
     84         elif self._iter:
     85             self._result_cache.extend(self._iter)

/home/wilfred/work/gap2/django/db/models/query.py in iterator(self)
    274         model = self.model
    275         compiler = self.query.get_compiler(using=db)
--> 276         for row in compiler.results_iter():
    277             if fill_cache:
    278                 obj, _ = get_cached_row(model, row,

/home/wilfred/work/gap2/djangotoolbox/db/basecompiler.py in results_iter(self)
    227         low_mark = self.query.low_mark
    228         high_mark = self.query.high_mark
--> 229         for entity in self.build_query(fields).fetch(low_mark, high_mark):
    230             yield self._make_result(entity, fields)
    231 

/home/wilfred/work/gap2/djangotoolbox/db/basecompiler.py in build_query(self, fields)
    287             fields = self.get_fields()
    288         query = self.query_class(self, fields)
--> 289         query.add_filters(self.query.where)
    290         query.order_by(self._get_ordering())
    291 

/home/wilfred/work/gap2/djangotoolbox/db/basecompiler.py in add_filters(self, filters)
     72         for child in children:
     73             if isinstance(child, Node):
---> 74                 self.add_filters(child)
     75                 continue
     76 

/home/wilfred/work/gap2/djangotoolbox/db/basecompiler.py in add_filters(self, filters)
     76 
     77             column, lookup_type, db_type, value = self._decode_child(child)
---> 78             self.add_filter(column, lookup_type, self._negated, db_type, value)
     79 
     80         if filters.negated:

/home/wilfred/work/gap2/djangoappengine/db/compiler.py in _func(*args, **kwargs)
     59     def _func(*args, **kwargs):
     60         try:
---> 61             return func(*args, **kwargs)
     62         except GAEError, e:
     63             raise DatabaseError, DatabaseError(str(e)), sys.exc_info()[2]

/home/wilfred/work/gap2/djangoappengine/db/compiler.py in add_filter(self, column, lookup_type, negated, db_type, value)
    241         elif lookup_type == 'in':
    242             # Create sub-query combinations, one for each value

--> 243             if len(self.gae_query) * len(value) > 30:
    244                 raise DatabaseError("You can't query against more than "
    245                                     "30 __in filter value combinations")

TypeError: object of type 'QueryWrapper' has no len()

[jonashaag]

Okay so we need a test in djangotoolbox for this and a fix in either djangoappengine or djangotoolbox, depending on which code is broken

[wrwrwr]

An exception should be raised if a subquery is used with the recent changes. However, we could try to evaluate subqueries or avoid them when constructing queries.