michaelhelmick / django-balancer

A set of tools for using Django's multi-db feature.
BSD 3-Clause "New" or "Revised" License
110 stars 5 forks source link

Can't do subqueries with queries on different DBs. #2

Closed bkonkle closed 14 years ago

bkonkle commented 14 years ago
  File "/path/to/my/code/django/db/models/query.py", line 112, in __nonzero__
    iter(self).next()

  File "/path/to/my/code/django/db/models/query.py", line 106, in _result_iter
    self._fill_cache()

  File "/path/to/my/code/django/db/models/query.py", line 760, in _fill_cache
    self._result_cache.append(self._iter.next())

  File "/path/to/my/code/django/db/models/query.py", line 269, in iterator
    for row in compiler.results_iter():

  File "/path/to/my/code/django/db/models/sql/compiler.py", line 672, in results_iter
    for rows in self.execute_sql(MULTI):

  File "/path/to/my/code/django/db/models/sql/compiler.py", line 717, in execute_sql
    sql, params = self.as_sql()

  File "/path/to/my/code/django/db/models/sql/compiler.py", line 65, in as_sql
    where, w_params = self.query.where.as_sql(qn=qn, connection=self.connection)

  File "/path/to/my/code/django/db/models/sql/where.py", line 91, in as_sql
    sql, params = child.as_sql(qn=qn, connection=connection)

  File "/path/to/my/code/django/db/models/sql/where.py", line 94, in as_sql
    sql, params = self.make_atom(child, qn, connection)

  File "/path/to/my/code/django/contrib/gis/db/models/sql/where.py", line 50, in make_atom
    return super(GeoWhereNode, self).make_atom(child, qn, connection)

  File "/path/to/my/code/django/db/models/sql/where.py", line 141, in make_atom
    lvalue, params = lvalue.process(lookup_type, params_or_value, connection)

  File "/path/to/my/code/django/db/models/sql/where.py", line 312, in process
    connection=connection, prepared=True)

  File "/path/to/my/code/django/db/models/fields/subclassing.py", line 53, in inner
    return func(*args, **kwargs)

  File "/path/to/my/code/django/db/models/fields/related.py", line 154, in get_db_prep_lookup
    sql, params = value._as_sql(connection=connection)

  File "/path/to/my/code/django/db/models/query.py", line 806, in _as_sql
    raise ValueError("Can't do subqueries with queries on different DBs.")

ValueError: Can't do subqueries with queries on different DBs.

Looks like it's related to the allow_relation method.

bkonkle commented 14 years ago

I created a test case for this here: http://github.com/bkonkle/django-balancer/commit/b68acef088181a0cdc4574cf2be7abf4783a72d7

Unfortunately, after more research it appears this is a hard and fast rule with QuerySet objects. If you're filtering a queryset by the results of another queryset, you'll need to explicitly use one of your databases for both querysets. I wasn't able to find a dynamic way around it.

tobiasmcnulty commented 12 years ago

For anyone else stumbling across this issue, a generic workaround is to import the global Django router (django.db.router) and call router.db_for_read() with one of the models in question. Then, pass that DB manually to both the queries.

bkonkle commented 12 years ago

Thanks for the workaround! I haven't touched this issue since 2010, but looking at it now it seems rather similar to #8. I think the request pinning feature that is being tested would help here, too.

tobiasmcnulty commented 12 years ago

Request pinning would definitely help here too. In a way it seems like a Django issue to me - shouldn't the query just be run on the database specified by the outer query, end of story?